[jira] [Commented] (CASSANDRA-8404) CQLSSTableLoader can not create SSTable for csv file of 10M rows.

2015-04-01 Thread Manish (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390048#comment-14390048
 ] 

Manish commented on CASSANDRA-8404:
---

I will check and let you know. 
I had faced this issue only on 32 bit Ubuntu OS and not on 64 bit Ubuntu OS. 
Since our QA and production machines are on 64 bit Ubuntu OS I did not face any 
issue in dev testing on QA machine. 


 CQLSSTableLoader can not create SSTable for csv file of 10M rows.
 -

 Key: CASSANDRA-8404
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8404
 Project: Cassandra
  Issue Type: Bug
 Environment: I am using Cassandra 2.1.1 on 32 bit Ubuntu 12.04. I am 
 running the program with -Xmx1000M
 manish@manish[~]: uname -a
 Linux manish 3.2.0-72-generic-pae #107-Ubuntu SMP Thu Nov 6 14:44:10 UTC 2014 
 i686 i686 i386 GNU/Linux
Reporter: Manish
 Fix For: 2.1.4

 Attachments: Test1.java, cassandra.yaml


 I am able to create SSTable for one file of 10M rows but not for other file. 
 The data file which works is subscribers1.gz and data file which does not 
 work is subscriber2.gz. Both files have same values in first column but 
 different values for second column. I wonder why CQLSSTableLoader does not 
 work for different set of data. 
 Program expected unzipped txt files. So please unzip files before running 
 program. What I have observed is High GC when program processes around 5.2M 
 lines of file subscriber2.gz. It is able to process till 5.8M lines with very 
 frequent Full GC runs. It is not able to process beyond 5.8M rows because of 
 memory not being available.
 I have attached Test1.java and cassandra.yaml I used for creating sstable. In 
 classpath I am specifying all jars of lib folder of extracted 
 apache-cassandra-2.1.1-bin.tar.gz 
 Jira does not allow a file of size greater than 10 MB. So I am sharing data 
 files in google drive.
 link to download subscribers1.gz
 https://drive.google.com/file/d/0B6_-ugKWlrfoOTRTa2FCNTFWU2c/view?usp=sharing
 link to download subscribers2.gz
 https://drive.google.com/file/d/0B6_-ugKWlrfocndycm9yM21rN0E/view?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8360) In DTCS, always compact SSTables in the same time window, even if they are fewer than min_threshold

2015-04-01 Thread Marcus Eriksson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson resolved CASSANDRA-8360.

Resolution: Fixed

will commit fix as part of CASSANDRA-8359

 In DTCS, always compact SSTables in the same time window, even if they are 
 fewer than min_threshold
 ---

 Key: CASSANDRA-8360
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8360
 Project: Cassandra
  Issue Type: Improvement
Reporter: Björn Hegerfors
Assignee: Björn Hegerfors
Priority: Minor
 Fix For: 3.0, 2.1.4, 2.0.14

 Attachments: cassandra-2.0-CASSANDRA-8360.txt


 DTCS uses min_threshold to decide how many time windows of the same size that 
 need to accumulate before merging into a larger window. The age of an SSTable 
 is determined as its min timestamp, and it always falls into exactly one of 
 the time windows. If multiple SSTables fall into the same window, DTCS 
 considers compacting them, but if they are fewer than min_threshold, it 
 decides not to do it.
 When do more than 1 but fewer than min_threshold SSTables end up in the same 
 time window (except for the current window), you might ask? In the current 
 state, DTCS can spill some extra SSTables into bigger windows when the 
 previous window wasn't fully compacted, which happens all the time when the 
 latest window stops being the current one. Also, repairs and hints can put 
 new SSTables in old windows.
 I think, and [~jjordan] agreed in a comment on CASSANDRA-6602, that DTCS 
 should ignore min_threshold and compact tables in the same windows regardless 
 of how few they are. I guess max_threshold should still be respected.
 [~jjordan] suggested that this should apply to all windows but the current 
 window, where all the new SSTables end up. That could make sense. I'm not 
 clear on whether compacting many SSTables at once is more cost efficient or 
 not, when it comes to the very newest and smallest SSTables. Maybe compacting 
 as soon as 2 SSTables are seen is fine if the initial window size is small 
 enough? I guess the opposite could be the case too; that the very newest 
 SSTables should be compacted very many at a time?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9084) Do not generate line number in logs

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390264#comment-14390264
 ] 

Benedict commented on CASSANDRA-9084:
-

Hmm. Yes, we shouldn't have that in the default logback config IMO. It's not 
like it helps tremendously anyway, since most log messages are unique to a 
single line of code. I'm not sure who's best placed to assess the logging. 
[~dbrosius]? Looks like you made the change to introduce logback, and also to 
introduce the %L - what are your thoughts?

 Do not generate line number in logs
 ---

 Key: CASSANDRA-9084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9084
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Andrey
Priority: Minor

 According to logback documentation 
 (http://logback.qos.ch/manual/layouts.html):
 {code}
 Generating the line number information is not particularly fast. Thus, its 
 use should be avoided unless execution speed is not an issue.
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-8360) In DTCS, always compact SSTables in the same time window, even if they are fewer than min_threshold

2015-04-01 Thread Marcus Eriksson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reopened CASSANDRA-8360:


reopening, seems now and base arguments are flipped in newestBucket

 In DTCS, always compact SSTables in the same time window, even if they are 
 fewer than min_threshold
 ---

 Key: CASSANDRA-8360
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8360
 Project: Cassandra
  Issue Type: Improvement
Reporter: Björn Hegerfors
Assignee: Björn Hegerfors
Priority: Minor
 Fix For: 3.0, 2.1.4, 2.0.14

 Attachments: cassandra-2.0-CASSANDRA-8360.txt


 DTCS uses min_threshold to decide how many time windows of the same size that 
 need to accumulate before merging into a larger window. The age of an SSTable 
 is determined as its min timestamp, and it always falls into exactly one of 
 the time windows. If multiple SSTables fall into the same window, DTCS 
 considers compacting them, but if they are fewer than min_threshold, it 
 decides not to do it.
 When do more than 1 but fewer than min_threshold SSTables end up in the same 
 time window (except for the current window), you might ask? In the current 
 state, DTCS can spill some extra SSTables into bigger windows when the 
 previous window wasn't fully compacted, which happens all the time when the 
 latest window stops being the current one. Also, repairs and hints can put 
 new SSTables in old windows.
 I think, and [~jjordan] agreed in a comment on CASSANDRA-6602, that DTCS 
 should ignore min_threshold and compact tables in the same windows regardless 
 of how few they are. I guess max_threshold should still be respected.
 [~jjordan] suggested that this should apply to all windows but the current 
 window, where all the new SSTables end up. That could make sense. I'm not 
 clear on whether compacting many SSTables at once is more cost efficient or 
 not, when it comes to the very newest and smallest SSTables. Maybe compacting 
 as soon as 2 SSTables are seen is fine if the initial window size is small 
 enough? I guess the opposite could be the case too; that the very newest 
 SSTables should be compacted very many at a time?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9056) Tombstoned SSTables are not removed past max_sstable_age_days when using DTCS

2015-04-01 Thread Marcus Eriksson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390472#comment-14390472
 ] 

Marcus Eriksson commented on CASSANDRA-9056:


i tweaked the dtest a bit, with this: 
https://github.com/krummas/cassandra-dtest/tree/marcuse/9056 and CASSANDRA-8359 
it works

 Tombstoned SSTables are not removed past max_sstable_age_days when using DTCS
 -

 Key: CASSANDRA-9056
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9056
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Shawn Kumar
Assignee: Marcus Eriksson
  Labels: compaction, dtcs

 When using DTCS, tombstoned sstables past max_sstable_age_days are not 
 removed by minor compactions. I was able to reproduce this manually and also 
 wrote a dtest (currently failing) which reproduces this issue: 
 [dtcs_deletion_test|https://github.com/riptano/cassandra-dtest/blob/master/compaction_test.py#L115]
  in compaction_test.py. I tried applying the patch in CASSANDRA-8359 but 
 found that the test still fails with the same issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Avoid race in cancelling compactions

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 0ba5f2721 - 730cc6064


Avoid race in cancelling compactions

Patch by marcuse; reviewed by yukim for CASSANDRA-9070


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2cf4ca3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2cf4ca3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2cf4ca3a

Branch: refs/heads/cassandra-2.1
Commit: 2cf4ca3a94ff972b422f1be1a2d601d07ff31a12
Parents: 3d0c4e7
Author: Marcus Eriksson marc...@apache.org
Authored: Tue Mar 31 11:05:40 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 14:32:47 2015 +0200

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cf4ca3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1afe6fe..b6b5caf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Avoid race in cancelling compactions (CASSANDRA-9070)
  * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  * Don't set clientMode to true when bulk-loading sstables to avoid
a NullPointerException (CASSANDRA-8238)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cf4ca3a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 6c6d3a2..9f7c8dd 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -145,6 +145,9 @@ public class CompactionTask extends AbstractCompactionTask
 collector.beginCompaction(ci);
 try
 {
+if (!controller.cfs.getCompactionStrategy().isActive)
+throw new 
CompactionInterruptedException(ci.getCompactionInfo());
+
 if (!iter.hasNext())
 {
 // don't mark compacted in the finally block, since if there 
_is_ nondeleted data,

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2015-04-01 Thread marcuse 
Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/CompactionTask.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/730cc606
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/730cc606
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/730cc606

Branch: refs/heads/cassandra-2.1
Commit: 730cc60640a683ef455c1b8856e4ed8fc39a460f
Parents: 0ba5f27 2cf4ca3
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 14:39:50 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 14:39:50 2015 +0200

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/730cc606/CHANGES.txt
--
diff --cc CHANGES.txt
index ab5fb2d,b6b5caf..3b99c61
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,70 -1,8 +1,71 @@@
 -2.0.14:
 +2.1.4
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys()

cassandra git commit: remove unnecessary anonymous class creation and generic parameters from NetworkTopologyStrategy

2015-04-01 Thread benedict 
Repository: cassandra
Updated Branches:
  refs/heads/trunk d7076a4b8 - 82a526863


remove unnecessary anonymous class creation and generic parameters from 
NetworkTopologyStrategy


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/82a52686
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/82a52686
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/82a52686

Branch: refs/heads/trunk
Commit: 82a526863b03f533318cfe00d74508cd3da235f3
Parents: d7076a4
Author: Benedict Elliott Smith bened...@apache.org
Authored: Wed Apr 1 13:57:43 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Wed Apr 1 13:57:43 2015 +0100

--
 .../locator/NetworkTopologyStrategy.java| 29 +---
 1 file changed, 13 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/82a52686/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
--
diff --git a/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java 
b/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
index b0d540c..fb291b5 100644
--- a/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
+++ b/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
@@ -81,13 +81,12 @@ public class NetworkTopologyStrategy extends 
AbstractReplicationStrategy
 public ListInetAddress calculateNaturalEndpoints(Token searchToken, 
TokenMetadata tokenMetadata)
 {
 // we want to preserve insertion order so that the first added 
endpoint becomes primary
-SetInetAddress replicas = new LinkedHashSetInetAddress();
+SetInetAddress replicas = new LinkedHashSet();
 // replicas we have found in each DC
-MapString, SetInetAddress dcReplicas = new HashMapString, 
SetInetAddress(datacenters.size())
-{{
-for (Map.EntryString, Integer dc : datacenters.entrySet())
-put(dc.getKey(), new HashSetInetAddress(dc.getValue()));
-}};
+MapString, SetInetAddress dcReplicas = new 
HashMap(datacenters.size());
+for (Map.EntryString, Integer dc : datacenters.entrySet())
+dcReplicas.put(dc.getKey(), new HashSet(dc.getValue()));
+
 Topology topology = tokenMetadata.getTopology();
 // all endpoints in each DC, so we can check when we have exhausted 
all the members of a DC
 MultimapString, InetAddress allEndpoints = 
topology.getDatacenterEndpoints();
@@ -96,18 +95,16 @@ public class NetworkTopologyStrategy extends 
AbstractReplicationStrategy
 assert !allEndpoints.isEmpty()  !racks.isEmpty() : not aware of any 
cluster members;
 
 // tracks the racks we have already placed replicas in
-MapString, SetString seenRacks = new HashMapString, 
SetString(datacenters.size())
-{{
-for (Map.EntryString, Integer dc : datacenters.entrySet())
-put(dc.getKey(), new HashSetString());
-}};
+MapString, SetString seenRacks = new HashMap(datacenters.size());
+for (Map.EntryString, Integer dc : datacenters.entrySet())
+seenRacks.put(dc.getKey(), new HashSet());
+
 // tracks the endpoints that we skipped over while looking for unique 
racks
 // when we relax the rack uniqueness we can append this to the current 
result so we don't have to wind back the iterator
-MapString, SetInetAddress skippedDcEndpoints = new HashMapString, 
SetInetAddress(datacenters.size())
-{{
-for (Map.EntryString, Integer dc : datacenters.entrySet())
-put(dc.getKey(), new LinkedHashSetInetAddress());
-}};
+MapString, SetInetAddress skippedDcEndpoints = new 
HashMap(datacenters.size());
+for (Map.EntryString, Integer dc : datacenters.entrySet())
+skippedDcEndpoints.put(dc.getKey(), new LinkedHashSet());
+
 IteratorToken tokenIter = 
TokenMetadata.ringIterator(tokenMetadata.sortedTokens(), searchToken, false);
 while (tokenIter.hasNext()  !hasSufficientReplicas(dcReplicas, 
allEndpoints))
 {

cassandra git commit: Avoid race in cancelling compactions

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 3d0c4e78c - 2cf4ca3a9


Avoid race in cancelling compactions

Patch by marcuse; reviewed by yukim for CASSANDRA-9070


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2cf4ca3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2cf4ca3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2cf4ca3a

Branch: refs/heads/cassandra-2.0
Commit: 2cf4ca3a94ff972b422f1be1a2d601d07ff31a12
Parents: 3d0c4e7
Author: Marcus Eriksson marc...@apache.org
Authored: Tue Mar 31 11:05:40 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 14:32:47 2015 +0200

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cf4ca3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1afe6fe..b6b5caf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Avoid race in cancelling compactions (CASSANDRA-9070)
  * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  * Don't set clientMode to true when bulk-loading sstables to avoid
a NullPointerException (CASSANDRA-8238)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cf4ca3a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 6c6d3a2..9f7c8dd 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -145,6 +145,9 @@ public class CompactionTask extends AbstractCompactionTask
 collector.beginCompaction(ci);
 try
 {
+if (!controller.cfs.getCompactionStrategy().isActive)
+throw new 
CompactionInterruptedException(ci.getCompactionInfo());
+
 if (!iter.hasNext())
 {
 // don't mark compacted in the finally block, since if there 
_is_ nondeleted data,

[jira] [Created] (CASSANDRA-9084) Do not generate line number in logs

2015-04-01 Thread Andrey (JIRA) 
Andrey created CASSANDRA-9084:
-

 Summary: Do not generate line number in logs
 Key: CASSANDRA-9084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9084
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Andrey
Priority: Minor


According to logback documentation (http://logback.qos.ch/manual/layouts.html):

{code}
Generating the line number information is not particularly fast. Thus, its use 
should be avoided unless execution speed is not an issue.
{code}






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2015-04-01 Thread marcuse 
Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/CompactionTask.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/730cc606
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/730cc606
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/730cc606

Branch: refs/heads/trunk
Commit: 730cc60640a683ef455c1b8856e4ed8fc39a460f
Parents: 0ba5f27 2cf4ca3
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 14:39:50 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 14:39:50 2015 +0200

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/730cc606/CHANGES.txt
--
diff --cc CHANGES.txt
index ab5fb2d,b6b5caf..3b99c61
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,70 -1,8 +1,71 @@@
 -2.0.14:
 +2.1.4
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and

[1/3] cassandra git commit: Avoid race in cancelling compactions

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/trunk 1b0a59e2a - d7076a4b8


Avoid race in cancelling compactions

Patch by marcuse; reviewed by yukim for CASSANDRA-9070


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2cf4ca3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2cf4ca3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2cf4ca3a

Branch: refs/heads/trunk
Commit: 2cf4ca3a94ff972b422f1be1a2d601d07ff31a12
Parents: 3d0c4e7
Author: Marcus Eriksson marc...@apache.org
Authored: Tue Mar 31 11:05:40 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 14:32:47 2015 +0200

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cf4ca3a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1afe6fe..b6b5caf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Avoid race in cancelling compactions (CASSANDRA-9070)
  * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  * Don't set clientMode to true when bulk-loading sstables to avoid
a NullPointerException (CASSANDRA-8238)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2cf4ca3a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 6c6d3a2..9f7c8dd 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -145,6 +145,9 @@ public class CompactionTask extends AbstractCompactionTask
 collector.beginCompaction(ci);
 try
 {
+if (!controller.cfs.getCompactionStrategy().isActive)
+throw new 
CompactionInterruptedException(ci.getCompactionInfo());
+
 if (!iter.hasNext())
 {
 // don't mark compacted in the finally block, since if there 
_is_ nondeleted data,

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-04-01 Thread marcuse 
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d7076a4b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d7076a4b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d7076a4b

Branch: refs/heads/trunk
Commit: d7076a4b8a210c5166c96fec793644950b7d772c
Parents: 1b0a59e 730cc60
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 14:39:56 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 14:39:56 2015 +0200

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/db/compaction/CompactionTask.java | 3 +++
 2 files changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7076a4b/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d7076a4b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 886ba34,392034c..0560328
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@@ -139,16 -137,10 +139,17 @@@ public class CompactionTask extends Abs
  // new sstables from flush can be added during a compaction, but only 
the compaction can remove them,
  // so in our single-threaded compaction world this is a valid way of 
determining if we're compacting
  // all the sstables (that existed when we started)
 -logger.info(Compacting {}, sstables);
 +StringBuilder ssTableLoggerMsg = new StringBuilder([);
 +for (SSTableReader sstr : sstables)
 +{
 +ssTableLoggerMsg.append(String.format(%s:level=%d, , 
sstr.getFilename(), sstr.getSSTableLevel()));
 +}
 +ssTableLoggerMsg.append(]);
 +String taskIdLoggerMsg = taskId == null ? 
UUIDGen.getTimeUUID().toString() : taskId.toString();
 +logger.info(Compacting ({}) {}, taskIdLoggerMsg, ssTableLoggerMsg);
  
  long start = System.nanoTime();
+ 
  long totalKeysWritten = 0;
  
  try (CompactionController controller = 
getCompactionController(sstables);)

[jira] [Commented] (CASSANDRA-9084) Do not generate line number in logs

2015-04-01 Thread Aleksey Yeschenko (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390490#comment-14390490
 ] 

Aleksey Yeschenko commented on CASSANDRA-9084:
--

Right. It's both not impacting performance much *and* isn't really useful, 
given uniqueness of the log messages and line numbers jumping between releases.

Can't have a strong opinion on this, but have a weak one - let's get rid of it.

 Do not generate line number in logs
 ---

 Key: CASSANDRA-9084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9084
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Andrey
Priority: Minor

 According to logback documentation 
 (http://logback.qos.ch/manual/layouts.html):
 {code}
 Generating the line number information is not particularly fast. Thus, its 
 use should be avoided unless execution speed is not an issue.
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390360#comment-14390360
 ] 

Benedict edited comment on CASSANDRA-8670 at 4/1/15 10:43 AM:
--

A little niggle was bugging me, and I decided to check it out, and I think it 
warrants further consideration:

The 
[windows|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/00cd9dc3c2b5/src/windows/native/java/net/SocketOutputStream.c]
 and 
[solaris|http://hg.openjdk.java.net/jdk6/jdk6/jdk/annotate/1a53516ce032/src/solaris/native/java/net/SocketOutputStream.c]
 implementations of SocketOutputStream allocate _temporary_ (not cached) heap 
(not Java heap) memory of the total size of the data being written to them, if 
they're above the size of the stack memory they allocate by default (according 
to the source, this ranges from 8K to 64K on 
[Solaris|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/4dbd83eb0250/src/solaris/native/java/net/net_util_md.h#l123],
 and 2K on 
[Windows|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/0c27202d66c1/src/windows/native/java/net/net_util_md.h#l236],
 for the stack buffer, with the heap buffer being a little larger)

This raises a few questions: 
# how much of our thread stack frame ends up being used by these methods? Seems 
we have some headroom with even the minimum stack frame, so maybe not worth 
worrying about; but larger writes will incur some non-java-heap allocations 
without much benefit
# should we simply copy our byte[] data into our direct buffer, and call write 
on this instead? this has the added advantage of avoiding the OutputStream 
entirely in situations where we have access to the Channel, which removes an 
entire codepath from the common execution path. There seems to be no downside, 
especially if our buffer is large enough, since the copying is exactly what the 
native code does anyway.
# what is behaviour on Linux? I can't for the life of me find the linux 
implementation; best I can assume is it is very similar to Solaris



was (Author: benedict):
A little niggle was bugging me, and I decided to check it out, and I think it 
warrants further consideration:

The 
[windows|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/00cd9dc3c2b5/src/windows/native/java/net/SocketOutputStream.c]
 and 
[solaris|http://hg.openjdk.java.net/jdk6/jdk6/jdk/annotate/1a53516ce032/src/solaris/native/java/net/SocketOutputStream.c]
 implementations of SocketOutputStream allocate heap (not Java heap) memory of 
the total size of the data being written to them, if they're above the size of 
the stack memory they allocate by default (according to the source, this ranges 
from 8K to 64K on 
[Solaris|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/4dbd83eb0250/src/solaris/native/java/net/net_util_md.h#l123],
 and 2K on 
[Windows|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/0c27202d66c1/src/windows/native/java/net/net_util_md.h#l236],
 for the stack buffer, with the heap buffer being a little larger)

This raises a few questions: 
# how much of our thread stack frame ends up being used by these methods? Seems 
we have some headroom with even the minimum stack frame, so maybe not worth 
worrying about; but larger writes will incur some non-java-heap allocations 
without much benefit
# should we simply copy our byte[] data into our direct buffer, and call write 
on this instead? this has the added advantage of avoiding the OutputStream 
entirely in situations where we have access to the Channel, which removes an 
entire codepath from the common execution path. There seems to be no downside, 
especially if our buffer is large enough, since the copying is exactly what the 
native code does anyway.
# what is behaviour on Linux? I can't for the life of me find the linux 
implementation; best I can assume is it is very similar to Solaris


 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause

[1/3] cassandra git commit: More aggressive check for expired sstables in DTCS

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/trunk 7f0a8542d - b09e39295


More aggressive check for expired sstables in DTCS

Patch by Björn Hegerfors; reviewed by marcuse for CASSANDRA-8359


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d0c4e78
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d0c4e78
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d0c4e78

Branch: refs/heads/trunk
Commit: 3d0c4e78c6bb3f8767aa0720b7de579908e2bf59
Parents: 74bfa77
Author: Björn Hegerfors bj...@spotify.com
Authored: Wed Apr 1 12:26:00 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 12:32:09 2015 +0200

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 31 ++
 .../DateTieredCompactionStrategyTest.java   | 43 
 3 files changed, 67 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d0c4e78/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a956eb6..1afe6fe 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  * Don't set clientMode to true when bulk-loading sstables to avoid
a NullPointerException (CASSANDRA-8238)
  * Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d0c4e78/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index 6b3e800..cfa9c8a 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -53,7 +53,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 
 while (true)
 {
-ListSSTableReader latestBucket = 
getNextBackgroundSStables(gcBefore);
+ListSSTableReader latestBucket = 
getNextBackgroundSSTables(gcBefore);
 
 if (latestBucket.isEmpty())
 return null;
@@ -68,24 +68,39 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
  * @param gcBefore
  * @return
  */
-private ListSSTableReader getNextBackgroundSStables(final int gcBefore)
+private ListSSTableReader getNextBackgroundSSTables(final int gcBefore)
 {
 if (!isEnabled() || cfs.getSSTables().isEmpty())
 return Collections.emptyList();
 
+SetSSTableReader uncompacting = cfs.getUncompactingSSTables();
+
+// Find fully expired SSTables. Those will be included no matter what.
+SetSSTableReader expired = 
CompactionController.getFullyExpiredSSTables(cfs, uncompacting, 
cfs.getOverlappingSSTables(uncompacting), gcBefore);
+SetSSTableReader candidates = 
Sets.newHashSet(filterSuspectSSTables(uncompacting));
+
+ListSSTableReader compactionCandidates = new 
ArrayList(getNextNonExpiredSSTables(Sets.difference(candidates, expired), 
gcBefore));
+if (!expired.isEmpty())
+{
+logger.debug(Including expired sstables: {}, expired);
+compactionCandidates.addAll(expired);
+}
+return compactionCandidates;
+}
+
+private ListSSTableReader 
getNextNonExpiredSSTables(IterableSSTableReader nonExpiringSSTables, final 
int gcBefore)
+{
 int base = cfs.getMinimumCompactionThreshold();
 long now = getNow();
 
-IterableSSTableReader candidates = 
filterSuspectSSTables(cfs.getUncompactingSSTables());
-
-ListSSTableReader mostInteresting = 
getCompactionCandidates(candidates, now, base);
+ListSSTableReader mostInteresting = 
getCompactionCandidates(nonExpiringSSTables, now, base);
 if (mostInteresting != null)
 return mostInteresting;
 
 // if there is no sstable to compact in standard way, try compacting 
single sstable whose droppable tombstone
 // ratio is greater than threshold.
 ListSSTableReader sstablesWithTombstones = Lists.newArrayList();
-for (SSTableReader sstable : candidates)
+for (SSTableReader sstable : nonExpiringSSTables)
 {
 if (worthDroppingTombstones(sstable, gcBefore))
 sstablesWithTombstones.add(sstable);
@@ -106,8 +121,8 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy

[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2015-04-01 Thread marcuse 
Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt

src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52191ebe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52191ebe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52191ebe

Branch: refs/heads/trunk
Commit: 52191ebe33d82bf89a57f019c026c88a8d82ee55
Parents: c9b83e0 3d0c4e7
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 12:34:01 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 12:56:21 2015 +0200

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 31 ++
 .../DateTieredCompactionStrategyTest.java   | 43 
 3 files changed, 67 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52191ebe/CHANGES.txt
--
diff --cc CHANGES.txt
index 7ded07b,1afe6fe..ab5fb2d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,69 -1,7 +1,70 @@@
 -2.0.14:
 +2.1.4
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write

[jira] [Commented] (CASSANDRA-9084) Do not generate line number in logs

2015-04-01 Thread Dave Brosius (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390420#comment-14390420
 ] 

Dave Brosius commented on CASSANDRA-9084:
-

i believe the format used was done to try to match what was done previously 
with log4j.

If you want to remove the line number, its fine with me. Altho if our standard 
configuration has log flooding where it would matter (other than startup), 
perhaps we should reduce that.

 Do not generate line number in logs
 ---

 Key: CASSANDRA-9084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9084
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Andrey
Priority: Minor

 According to logback documentation 
 (http://logback.qos.ch/manual/layouts.html):
 {code}
 Generating the line number information is not particularly fast. Thus, its 
 use should be avoided unless execution speed is not an issue.
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9084) Do not generate line number in logs

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390447#comment-14390447
 ] 

Benedict commented on CASSANDRA-9084:
-

[~jbellis] [~iamaleksey]? Thoughts? It's clearly not a bottleneck, but it is 
also probably superfluous. Although it can server as corroboration you're 
looking at the right source version when debugging log outputs. I don't have a 
particularly strong opinion either way.

 Do not generate line number in logs
 ---

 Key: CASSANDRA-9084
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9084
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Andrey
Priority: Minor

 According to logback documentation 
 (http://logback.qos.ch/manual/layouts.html):
 {code}
 Generating the line number information is not particularly fast. Thus, its 
 use should be avoided unless execution speed is not an issue.
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Benedict (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict reopened CASSANDRA-8670:
-

A little niggle was bugging me, and I decided to check it out, and I think it 
warrants further consideration:

The 
[windows|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/00cd9dc3c2b5/src/windows/native/java/net/SocketOutputStream.c]
 and 
[solaris|http://hg.openjdk.java.net/jdk6/jdk6/jdk/annotate/1a53516ce032/src/solaris/native/java/net/SocketOutputStream.c]
 implementations of SocketOutputStream allocate heap (not Java heap) memory of 
the total size of the data being written to them, if they're above the size of 
the stack memory they allocate by default (according to the source, this ranges 
from 8K to 64K on 
[Solaris|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/4dbd83eb0250/src/solaris/native/java/net/net_util_md.h#l123],
 and 2K on 
[Windows|http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/0c27202d66c1/src/windows/native/java/net/net_util_md.h#l236],
 for the stack buffer, with the heap buffer being a little larger)

This raises a few questions: 
# how much of our thread stack frame ends up being used by these methods? Seems 
we have some headroom with even the minimum stack frame, so maybe not worth 
worrying about; but larger writes will incur some non-java-heap allocations 
without much benefit
# should we simply copy our byte[] data into our direct buffer, and call write 
on this instead? this has the added advantage of avoiding the OutputStream 
entirely in situations where we have access to the Channel, which removes an 
entire codepath from the common execution path. There seems to be no downside, 
especially if our buffer is large enough, since the copying is exactly what the 
native code does anyway.
# what is behaviour on Linux? I can't for the life of me find the linux 
implementation; best I can assume is it is very similar to Solaris


 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7486) Compare CMS and G1 pause times

2015-04-01 Thread Robert Stupp (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390359#comment-14390359
 ] 

Robert Stupp commented on CASSANDRA-7486:
-

Nice :)

Looking forward to see Oracle JVM and C* 2.1 results. (TBH, I don't expect much 
difference between OpenJDK8 and Oracle JDK8)

But more interesting would be how G1 behaves w/ read and mixed workloads.


 Compare CMS and G1 pause times
 --

 Key: CASSANDRA-7486
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7486
 Project: Cassandra
  Issue Type: Test
  Components: Config
Reporter: Jonathan Ellis
Assignee: Shawn Kumar
 Fix For: 2.1.4


 See 
 http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning
  and https://twitter.com/rbranson/status/482113561431265281
 May want to default 2.1 to G1.
 2.1 is a different animal from 2.0 after moving most of memtables off heap.  
 Suspect this will help G1 even more than CMS.  (NB this is off by default but 
 needs to be part of the test.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Robert Stupp (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390400#comment-14390400
 ] 

Robert Stupp commented on CASSANDRA-8670:
-

Your assumption is correct 
{{jdk/src/solaris/native/java/net/SocketOutputStream.c}} is the Linux source.
What I don't understand is why 
{{Java_java_net_SocketOutputStream_socketWrite0}} not just uses the given 
{{byte[]}} but copies to a stack/heap buffer.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390405#comment-14390405
 ] 

Benedict commented on CASSANDRA-8670:
-

bq. What I don't understand is why 
Java_java_net_SocketOutputStream_socketWrite0 not just uses the given byte[] 
but copies to a stack/heap buffer.

Time to safe point. Using it directly would require pausing GC until the socket 
write had completed, which could be lengthy (since it's blocking, it's 
unbounded, in fact)

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-04-01 Thread marcuse 
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b0a59e2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b0a59e2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b0a59e2

Branch: refs/heads/trunk
Commit: 1b0a59e2ac5c29176280649e65ff4fe82bb76281
Parents: b09e392 0ba5f27
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 13:14:45 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 13:14:45 2015 +0200

--
 .../cassandra/db/compaction/DateTieredCompactionStrategy.java  | 2 +-
 .../cassandra/db/compaction/DateTieredCompactionStrategyTest.java  | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b0a59e2/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b0a59e2/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--

[jira] [Commented] (CASSANDRA-8672) Ambiguous WriteTimeoutException while completing pending CAS commits

2015-04-01 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390450#comment-14390450
 ] 

Sylvain Lebresne commented on CASSANDRA-8672:
-

bq. Its therefor not safe to assume that successive CAS or SERIAL read 
operations will cause a (write-)timeouted CAS operation to get eventually 
applied.

The only case where you can assume that is if the timeout happens during the 
{{commitPaxos}} phase. But we already distinguish timeouts in that case since 
they have a {{WriteType.SIMPLE}}.

bq. However, as theres a chance that the timeout occurred while writing the 
actual CAS operation, another write could potentially complete it and our CAS 
condition will get a different result upon retry.

Yes, that *is* a problem you have to deal with at the application level. But no 
amount of disambiguation of {{WriteTimeoutException}} will remove this: if 
there is a timeout during the propose phase, we just don't know if the write 
will be eventually applied or not. In other words, in many cases, the retry 
mechanism that you'll have to implement client side will involve a SERIAL read 
to figure out if the write was applied or not.

bq. I think the best option is to add a new {{WriteType.CAS_PREPARE}}

We could add a different {{WriteType}} if the timeout is during the prepare 
phase, and client could have an easier time retrying in that case since they 
can assume the update hasn't been applied, but as said above, this would change 
the fact that you will have to handle timeouts during the propose phase the 
hard way. And given that you have to do the latter, I wonder how meaningful it 
is to optimize for the former. I also don't know if that kind of subtle 
difference will still make sense post-CASSANDRA-6246. So I'm not strongly 
opposed, but I do wonder if exposing that kind of subtlety won't confuse users 
more than it will help and/or have a short shelf life (due to CASSANDRA-6246)

 Ambiguous WriteTimeoutException while completing pending CAS commits
 

 Key: CASSANDRA-8672
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8672
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Stefan Podkowinski
Assignee: Tyler Hobbs
Priority: Minor
  Labels: CAS
 Fix For: 3.0


 Any CAS update has a chance to trigger a pending/stalled commit of any 
 previously agreed on CAS update. After completing the pending commit, the CAS 
 operation will resume to execute the actual update and also possibly create a 
 new commit. See StorageProxy.cas()
 Theres two possbile execution paths that might end up throwing a 
 WriteTimeoutException:
 cas() - beginAndRepairPaxos() - commitPaxos()
 cas() - commitPaxos()
 Unfortunatelly clients catching a WriteTimeoutException won't be able to tell 
 at which stage the commit failed. My guess would be that most developers are 
 not aware that the beginAndRepairPaxos() could also trigger a write and 
 assume that write timeouts would refer to a timeout while writting the actual 
 CAS update. Its therefor not safe to assume that successive CAS or SERIAL 
 read operations will cause a (write-)timeouted CAS operation to get 
 eventually applied. Although some [best-practices 
 advise|http://www.datastax.com/dev/blog/cassandra-error-handling-done-right] 
 claims otherwise.
 At this point the safest bet is possibly to retry the complete business 
 transaction in case of an WriteTimeoutException. However, as theres a chance 
 that the timeout occurred while writing the actual CAS operation, another 
 write could potentially complete it and our CAS condition will get a 
 different result upon retry.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8099) Refactor and modernize the storage engine

2015-04-01 Thread Robert Stupp (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390361#comment-14390361
 ] 

Robert Stupp commented on CASSANDRA-8099:
-

Just found some naming issues and some nits.

Altogether I have to say, that CASSANDRA-8099 is a great step forward! It 
simplifies a lot of areas in the code, explains a lot of things in javadoc and 
makes it a lot easier to follow the travelled code path using (mostly ;) ) 
appropriate nomenclature.

*NITs*

* {{org.apache.cassandra.db.ReadQuery}} can be an interface (only abstract 
methods) and is mentioned as an interface in the javadoc ;)
* {{org.apache.cassandra.config.CFMetaData#columnMetadata}} can be final
* {{org.apache.cassandra.config.CFMetaData#getDefaultIndexName}}, 
{{#isNameValid}}, {{#isIndexNameValid}} use non-compiled regexp

I didn’t create a pull-req since all findings above are just nits and could 
only make final rebasing harder.

*Nomenclature*

The name {{ColumnFilter}} is a bit misleading. From the first impression I 
thought it’s a filter that filters CQL columns - but it’s used to to a 2i 
lookup.

Can you rename {{NamesPartitionFilter}} to something with _clustering key_ in 
it? I know that the term _name_ is used elsewhere for clustering key.

{{CBuilder}}/{{MultiCBuilder}} could be more expressive as 
{{ClusteringBuilder}}/{{MultipleClusteringsBuilder}}

*Misc*

The first time I ran into a situation where {{cluster_name}} and {{host_id}} 
were null in {{system.local}}. But had no luck reproducing this (I’m sure I did 
a {{ant realclean jar}} and {{rm -rf data/*}} before). So just take this as a 
note - not something worth to discuss.

I did a quickdirty prototype of CASSANDRA-7396 based on 8099 and it looks much 
easier (without the slicing stuff).

 Refactor and modernize the storage engine
 -

 Key: CASSANDRA-8099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8099
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 3.0

 Attachments: 8099-nit


 The current storage engine (which for this ticket I'll loosely define as the 
 code implementing the read/write path) is suffering from old age. One of the 
 main problem is that the only structure it deals with is the cell, which 
 completely ignores the more high level CQL structure that groups cell into 
 (CQL) rows.
 This leads to many inefficiencies, like the fact that during a reads we have 
 to group cells multiple times (to count on replica, then to count on the 
 coordinator, then to produce the CQL resultset) because we forget about the 
 grouping right away each time (so lots of useless cell names comparisons in 
 particular). But outside inefficiencies, having to manually recreate the CQL 
 structure every time we need it for something is hindering new features and 
 makes the code more complex that it should be.
 Said storage engine also has tons of technical debt. To pick an example, the 
 fact that during range queries we update {{SliceQueryFilter.count}} is pretty 
 hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has 
 to go into to simply remove the last query result.
 So I want to bite the bullet and modernize this storage engine. I propose to 
 do 2 main things:
 # Make the storage engine more aware of the CQL structure. In practice, 
 instead of having partitions be a simple iterable map of cells, it should be 
 an iterable list of row (each being itself composed of per-column cells, 
 though obviously not exactly the same kind of cell we have today).
 # Make the engine more iterative. What I mean here is that in the read path, 
 we end up reading all cells in memory (we put them in a ColumnFamily object), 
 but there is really no reason to. If instead we were working with iterators 
 all the way through, we could get to a point where we're basically 
 transferring data from disk to the network, and we should be able to reduce 
 GC substantially.
 Please note that such refactor should provide some performance improvements 
 right off the bat but it's not it's primary goal either. It's primary goal is 
 to simplify the storage engine and adds abstraction that are better suited to 
 further optimizations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2015-04-01 Thread marcuse 
Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt

src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/52191ebe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/52191ebe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/52191ebe

Branch: refs/heads/cassandra-2.1
Commit: 52191ebe33d82bf89a57f019c026c88a8d82ee55
Parents: c9b83e0 3d0c4e7
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 12:34:01 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 12:56:21 2015 +0200

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 31 ++
 .../DateTieredCompactionStrategyTest.java   | 43 
 3 files changed, 67 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/52191ebe/CHANGES.txt
--
diff --cc CHANGES.txt
index 7ded07b,1afe6fe..ab5fb2d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,69 -1,7 +1,70 @@@
 -2.0.14:
 +2.1.4
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + *

[1/2] cassandra git commit: More aggressive check for expired sstables in DTCS

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 c9b83e0a8 - 52191ebe3


More aggressive check for expired sstables in DTCS

Patch by Björn Hegerfors; reviewed by marcuse for CASSANDRA-8359


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d0c4e78
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d0c4e78
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d0c4e78

Branch: refs/heads/cassandra-2.1
Commit: 3d0c4e78c6bb3f8767aa0720b7de579908e2bf59
Parents: 74bfa77
Author: Björn Hegerfors bj...@spotify.com
Authored: Wed Apr 1 12:26:00 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 12:32:09 2015 +0200

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 31 ++
 .../DateTieredCompactionStrategyTest.java   | 43 
 3 files changed, 67 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d0c4e78/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a956eb6..1afe6fe 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  * Don't set clientMode to true when bulk-loading sstables to avoid
a NullPointerException (CASSANDRA-8238)
  * Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d0c4e78/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index 6b3e800..cfa9c8a 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -53,7 +53,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 
 while (true)
 {
-ListSSTableReader latestBucket = 
getNextBackgroundSStables(gcBefore);
+ListSSTableReader latestBucket = 
getNextBackgroundSSTables(gcBefore);
 
 if (latestBucket.isEmpty())
 return null;
@@ -68,24 +68,39 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
  * @param gcBefore
  * @return
  */
-private ListSSTableReader getNextBackgroundSStables(final int gcBefore)
+private ListSSTableReader getNextBackgroundSSTables(final int gcBefore)
 {
 if (!isEnabled() || cfs.getSSTables().isEmpty())
 return Collections.emptyList();
 
+SetSSTableReader uncompacting = cfs.getUncompactingSSTables();
+
+// Find fully expired SSTables. Those will be included no matter what.
+SetSSTableReader expired = 
CompactionController.getFullyExpiredSSTables(cfs, uncompacting, 
cfs.getOverlappingSSTables(uncompacting), gcBefore);
+SetSSTableReader candidates = 
Sets.newHashSet(filterSuspectSSTables(uncompacting));
+
+ListSSTableReader compactionCandidates = new 
ArrayList(getNextNonExpiredSSTables(Sets.difference(candidates, expired), 
gcBefore));
+if (!expired.isEmpty())
+{
+logger.debug(Including expired sstables: {}, expired);
+compactionCandidates.addAll(expired);
+}
+return compactionCandidates;
+}
+
+private ListSSTableReader 
getNextNonExpiredSSTables(IterableSSTableReader nonExpiringSSTables, final 
int gcBefore)
+{
 int base = cfs.getMinimumCompactionThreshold();
 long now = getNow();
 
-IterableSSTableReader candidates = 
filterSuspectSSTables(cfs.getUncompactingSSTables());
-
-ListSSTableReader mostInteresting = 
getCompactionCandidates(candidates, now, base);
+ListSSTableReader mostInteresting = 
getCompactionCandidates(nonExpiringSSTables, now, base);
 if (mostInteresting != null)
 return mostInteresting;
 
 // if there is no sstable to compact in standard way, try compacting 
single sstable whose droppable tombstone
 // ratio is greater than threshold.
 ListSSTableReader sstablesWithTombstones = Lists.newArrayList();
-for (SSTableReader sstable : candidates)
+for (SSTableReader sstable : nonExpiringSSTables)
 {
 if (worthDroppingTombstones(sstable, gcBefore))
 sstablesWithTombstones.add(sstable);
@@ -106,8 +121,8 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-04-01 Thread marcuse 
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b09e3929
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b09e3929
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b09e3929

Branch: refs/heads/trunk
Commit: b09e39295cdd26690a7d6f9a7c5417c26e9376fc
Parents: 7f0a854 52191eb
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 12:56:34 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 12:56:34 2015 +0200

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 31 ++
 .../DateTieredCompactionStrategyTest.java   | 43 
 3 files changed, 67 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b09e3929/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b09e3929/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b09e3929/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--

cassandra git commit: More aggressive check for expired sstables in DTCS

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 74bfa773a - 3d0c4e78c


More aggressive check for expired sstables in DTCS

Patch by Björn Hegerfors; reviewed by marcuse for CASSANDRA-8359


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d0c4e78
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d0c4e78
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d0c4e78

Branch: refs/heads/cassandra-2.0
Commit: 3d0c4e78c6bb3f8767aa0720b7de579908e2bf59
Parents: 74bfa77
Author: Björn Hegerfors bj...@spotify.com
Authored: Wed Apr 1 12:26:00 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 12:32:09 2015 +0200

--
 CHANGES.txt |  1 +
 .../DateTieredCompactionStrategy.java   | 31 ++
 .../DateTieredCompactionStrategyTest.java   | 43 
 3 files changed, 67 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d0c4e78/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a956eb6..1afe6fe 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  * Don't set clientMode to true when bulk-loading sstables to avoid
a NullPointerException (CASSANDRA-8238)
  * Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d0c4e78/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index 6b3e800..cfa9c8a 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -53,7 +53,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 
 while (true)
 {
-ListSSTableReader latestBucket = 
getNextBackgroundSStables(gcBefore);
+ListSSTableReader latestBucket = 
getNextBackgroundSSTables(gcBefore);
 
 if (latestBucket.isEmpty())
 return null;
@@ -68,24 +68,39 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
  * @param gcBefore
  * @return
  */
-private ListSSTableReader getNextBackgroundSStables(final int gcBefore)
+private ListSSTableReader getNextBackgroundSSTables(final int gcBefore)
 {
 if (!isEnabled() || cfs.getSSTables().isEmpty())
 return Collections.emptyList();
 
+SetSSTableReader uncompacting = cfs.getUncompactingSSTables();
+
+// Find fully expired SSTables. Those will be included no matter what.
+SetSSTableReader expired = 
CompactionController.getFullyExpiredSSTables(cfs, uncompacting, 
cfs.getOverlappingSSTables(uncompacting), gcBefore);
+SetSSTableReader candidates = 
Sets.newHashSet(filterSuspectSSTables(uncompacting));
+
+ListSSTableReader compactionCandidates = new 
ArrayList(getNextNonExpiredSSTables(Sets.difference(candidates, expired), 
gcBefore));
+if (!expired.isEmpty())
+{
+logger.debug(Including expired sstables: {}, expired);
+compactionCandidates.addAll(expired);
+}
+return compactionCandidates;
+}
+
+private ListSSTableReader 
getNextNonExpiredSSTables(IterableSSTableReader nonExpiringSSTables, final 
int gcBefore)
+{
 int base = cfs.getMinimumCompactionThreshold();
 long now = getNow();
 
-IterableSSTableReader candidates = 
filterSuspectSSTables(cfs.getUncompactingSSTables());
-
-ListSSTableReader mostInteresting = 
getCompactionCandidates(candidates, now, base);
+ListSSTableReader mostInteresting = 
getCompactionCandidates(nonExpiringSSTables, now, base);
 if (mostInteresting != null)
 return mostInteresting;
 
 // if there is no sstable to compact in standard way, try compacting 
single sstable whose droppable tombstone
 // ratio is greater than threshold.
 ListSSTableReader sstablesWithTombstones = Lists.newArrayList();
-for (SSTableReader sstable : candidates)
+for (SSTableReader sstable : nonExpiringSSTables)
 {
 if (worthDroppingTombstones(sstable, gcBefore))
 sstablesWithTombstones.add(sstable);
@@ -106,8 +121,8 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy

[1/2] cassandra git commit: Fix merge from CASSANDRA-8359

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/trunk b09e39295 - 1b0a59e2a


Fix merge from CASSANDRA-8359


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0ba5f272
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0ba5f272
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0ba5f272

Branch: refs/heads/trunk
Commit: 0ba5f2721b8d6b57592532654d8533acb9964807
Parents: 52191eb
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 13:14:40 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 13:14:40 2015 +0200

--
 .../cassandra/db/compaction/DateTieredCompactionStrategy.java  | 2 +-
 .../cassandra/db/compaction/DateTieredCompactionStrategyTest.java  | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5f272/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index 7841c93..d955ebf 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -74,7 +74,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 if (!isEnabled() || cfs.getSSTables().isEmpty())
 return Collections.emptyList();
 
-SetSSTableReader uncompacting = cfs.getUncompactingSSTables();
+SetSSTableReader uncompacting = Sets.intersection(sstables, 
cfs.getUncompactingSSTables());
 
 // Find fully expired SSTables. Those will be included no matter what.
 SetSSTableReader expired = 
CompactionController.getFullyExpiredSSTables(cfs, uncompacting, 
cfs.getOverlappingSSTables(uncompacting), gcBefore);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5f272/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
index becddeb..743a337 100644
--- 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
@@ -306,6 +306,8 @@ public class DateTieredCompactionStrategyTest extends 
SchemaLoader
 
options.put(DateTieredCompactionStrategyOptions.TIMESTAMP_RESOLUTION_KEY, 
MILLISECONDS);
 options.put(DateTieredCompactionStrategyOptions.MAX_SSTABLE_AGE_KEY, 
Double.toString((1d / (24 * 60 * 60;
 DateTieredCompactionStrategy dtcs = new 
DateTieredCompactionStrategy(cfs, options);
+for (SSTableReader sstable : cfs.getSSTables())
+dtcs.addSSTable(sstable);
 dtcs.startup();
 assertNull(dtcs.getNextBackgroundTask((int) 
(System.currentTimeMillis() / 1000)));
 Thread.sleep(7000);

cassandra git commit: Fix merge from CASSANDRA-8359

2015-04-01 Thread marcuse 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 52191ebe3 - 0ba5f2721


Fix merge from CASSANDRA-8359


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0ba5f272
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0ba5f272
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0ba5f272

Branch: refs/heads/cassandra-2.1
Commit: 0ba5f2721b8d6b57592532654d8533acb9964807
Parents: 52191eb
Author: Marcus Eriksson marc...@apache.org
Authored: Wed Apr 1 13:14:40 2015 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Wed Apr 1 13:14:40 2015 +0200

--
 .../cassandra/db/compaction/DateTieredCompactionStrategy.java  | 2 +-
 .../cassandra/db/compaction/DateTieredCompactionStrategyTest.java  | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5f272/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
index 7841c93..d955ebf 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java
@@ -74,7 +74,7 @@ public class DateTieredCompactionStrategy extends 
AbstractCompactionStrategy
 if (!isEnabled() || cfs.getSSTables().isEmpty())
 return Collections.emptyList();
 
-SetSSTableReader uncompacting = cfs.getUncompactingSSTables();
+SetSSTableReader uncompacting = Sets.intersection(sstables, 
cfs.getUncompactingSSTables());
 
 // Find fully expired SSTables. Those will be included no matter what.
 SetSSTableReader expired = 
CompactionController.getFullyExpiredSSTables(cfs, uncompacting, 
cfs.getOverlappingSSTables(uncompacting), gcBefore);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ba5f272/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
index becddeb..743a337 100644
--- 
a/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
+++ 
b/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java
@@ -306,6 +306,8 @@ public class DateTieredCompactionStrategyTest extends 
SchemaLoader
 
options.put(DateTieredCompactionStrategyOptions.TIMESTAMP_RESOLUTION_KEY, 
MILLISECONDS);
 options.put(DateTieredCompactionStrategyOptions.MAX_SSTABLE_AGE_KEY, 
Double.toString((1d / (24 * 60 * 60;
 DateTieredCompactionStrategy dtcs = new 
DateTieredCompactionStrategy(cfs, options);
+for (SSTableReader sstable : cfs.getSSTables())
+dtcs.addSSTable(sstable);
 dtcs.startup();
 assertNull(dtcs.getNextBackgroundTask((int) 
(System.currentTimeMillis() / 1000)));
 Thread.sleep(7000);

svn commit: r8503 - in /release/cassandra: 2.0.14/ 2.1.4/ debian/dists/20x/ debian/dists/20x/main/binary-amd64/ debian/dists/20x/main/binary-i386/ debian/dists/20x/main/source/ debian/dists/21x/ debia

2015-04-01 Thread jake 
Author: jake
Date: Wed Apr  1 11:51:08 2015
New Revision: 8503

Log:
new release

Added:
release/cassandra/2.0.14/
release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz   (with props)
release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.asc
release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.asc.md5
release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.asc.sha1
release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.md5
release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.sha1
release/cassandra/2.0.14/apache-cassandra-2.0.14-src.tar.gz   (with props)
release/cassandra/2.0.14/apache-cassandra-2.0.14-src.tar.gz.asc
release/cassandra/2.0.14/apache-cassandra-2.0.14-src.tar.gz.asc.md5
release/cassandra/2.0.14/apache-cassandra-2.0.14-src.tar.gz.asc.sha1
release/cassandra/2.0.14/apache-cassandra-2.0.14-src.tar.gz.md5
release/cassandra/2.0.14/apache-cassandra-2.0.14-src.tar.gz.sha1
release/cassandra/2.1.4/
release/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz   (with props)
release/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.asc
release/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.asc.md5
release/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.asc.sha1
release/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.md5
release/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.sha1
release/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz   (with props)
release/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.asc
release/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.asc.md5
release/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.asc.sha1
release/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.md5
release/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.sha1

release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_2.1.4_all.deb   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.14.diff.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.14.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.14.orig.tar.gz 
  (with props)

release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.14.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.0.14_all.deb   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.4.diff.gz   
(with props)
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.4.dsc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.4.orig.tar.gz  
 (with props)

release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.4.orig.tar.gz.asc
release/cassandra/debian/pool/main/c/cassandra/cassandra_2.1.4_all.deb   
(with props)
Modified:
release/cassandra/debian/dists/20x/InRelease
release/cassandra/debian/dists/20x/Release
release/cassandra/debian/dists/20x/Release.gpg
release/cassandra/debian/dists/20x/main/binary-amd64/Packages
release/cassandra/debian/dists/20x/main/binary-amd64/Packages.gz
release/cassandra/debian/dists/20x/main/binary-i386/Packages
release/cassandra/debian/dists/20x/main/binary-i386/Packages.gz
release/cassandra/debian/dists/20x/main/source/Sources.gz
release/cassandra/debian/dists/21x/InRelease
release/cassandra/debian/dists/21x/Release
release/cassandra/debian/dists/21x/Release.gpg
release/cassandra/debian/dists/21x/main/binary-amd64/Packages
release/cassandra/debian/dists/21x/main/binary-amd64/Packages.gz
release/cassandra/debian/dists/21x/main/binary-i386/Packages
release/cassandra/debian/dists/21x/main/binary-i386/Packages.gz
release/cassandra/debian/dists/21x/main/source/Sources.gz

Added: release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz
==
Binary file - no diff available.

Propchange: release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.asc
==
--- release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.asc (added)
+++ release/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.asc Wed Apr  1 
11:51:08 2015
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQIcBAABAgAGBQJVGVvqAAoJEHSdbuwDU7Escw0QAKQd35s7OZGI05ZjtLiW43Cy
+jX/UoSjtHqXfTjB73dDl9gK9gUxKCa6qWH5LIYIWpNunNGNZUt+mWbYYX2ihVwCe
+vR/7CsUO4a7aaJYYsU5R7rO9B6qOxm5ZGCbFvBkUpFhl5LYpwFechWA7sC5GyAaw
+AU07xpus7XTRXWfaA0/VH7PYjvfn2Caarfbl65qA+3X9R3LSrV5YxU+L8zbVOA/7
+ZUc6MOGW6t1H80Pswa5I5YZZwmTXpCnrBe1nDPyHglDvVnLGYK1kwDNxoYmB3H5J
+wfOdYt2L/E+fqEUSUGy03U22hb6p9bN1ubxGhii7piBsqXlXG9w+i6KdnYH2t4Vi

[jira] [Commented] (CASSANDRA-8672) Ambiguous WriteTimeoutException while completing pending CAS commits

2015-04-01 Thread Stefan Podkowinski (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390526#comment-14390526
 ] 

Stefan Podkowinski commented on CASSANDRA-8672:
---

What is the reason of having `WriteType.CAS` if I have to retry using serial 
reads for both CAS and SIMPLE timeouts? I don't see the point of implementing 
different error handling cases based on this flag given the current 
implementation. 

With the new prepare flag the error handling could be done like this:
CAS_PREPARE - retry cas write operation
CAS - serial read and retry write if needed
SIMPLE - ignore, will be committed later
 


 Ambiguous WriteTimeoutException while completing pending CAS commits
 

 Key: CASSANDRA-8672
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8672
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Stefan Podkowinski
Assignee: Tyler Hobbs
Priority: Minor
  Labels: CAS
 Fix For: 3.0


 Any CAS update has a chance to trigger a pending/stalled commit of any 
 previously agreed on CAS update. After completing the pending commit, the CAS 
 operation will resume to execute the actual update and also possibly create a 
 new commit. See StorageProxy.cas()
 Theres two possbile execution paths that might end up throwing a 
 WriteTimeoutException:
 cas() - beginAndRepairPaxos() - commitPaxos()
 cas() - commitPaxos()
 Unfortunatelly clients catching a WriteTimeoutException won't be able to tell 
 at which stage the commit failed. My guess would be that most developers are 
 not aware that the beginAndRepairPaxos() could also trigger a write and 
 assume that write timeouts would refer to a timeout while writting the actual 
 CAS update. Its therefor not safe to assume that successive CAS or SERIAL 
 read operations will cause a (write-)timeouted CAS operation to get 
 eventually applied. Although some [best-practices 
 advise|http://www.datastax.com/dev/blog/cassandra-error-handling-done-right] 
 claims otherwise.
 At this point the safest bet is possibly to retry the complete business 
 transaction in case of an WriteTimeoutException. However, as theres a chance 
 that the timeout occurred while writing the actual CAS operation, another 
 write could potentially complete it and our CAS condition will get a 
 different result upon retry.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390561#comment-14390561
 ] 

Benedict commented on CASSANDRA-8670:
-

bq. In most cases we opt to use the channel when available. ... Where we could 
really do better is when using compression

You're right. I thought we had more paths leftover, but it is just compression 
really. That's a much cleaner state of affairs!

It looks like compression will typically be under the stack buffer size, so we 
don't have such a problem, but it would still be nice to move to a 
DirectByteBuffer compressed output stream. It should be quite viable since 
there are now compression methods for working over these directly, but that 
should probably be a follow up ticket. Do you want to file, or shall I?

bq. BufferedDataOutputStreamPlus.close() will clean the buffer even if it might 
not own the buffer such as when it is provided to the constructor. It also 
doesn't check if the channel is null.

We don't call close in situations where this is a problem

bq. When used from the commit log without a channel it will throw an NPE if it 
exceeds the capacity of the buffer when it goes to flush

We presize the buffer correctly

Still, on both these counts we could offer better safety if we wanted, without 
much cost. If we used a DataOutputBuffer it would solve these problems, and we 
could assert that the final buffer is == the provided buffer...

bq. It almost seems like there is a case for a version just for wrapping fixed 
size bytebuffers.

Possibly. I'm on the fence, since it's extra code cache and class hierarchy 
pollution, with limited positive impact. We would need to duplicate the whole 
of BufferedDataOutputStreamPlus. You could benchmark to see if there's an 
appreciable difference? If the difference is small, I would prefer to avoid the 
pollution.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390561#comment-14390561
 ] 

Benedict edited comment on CASSANDRA-8670 at 4/1/15 1:46 PM:
-

bq. In most cases we opt to use the channel when available. ... Where we could 
really do better is when using compression

You're right. I thought we had more paths leftover, but it is just compression 
really. That's a much cleaner state of affairs!

It looks like compression will typically be under the stack buffer size, so we 
don't have such a problem, but it would still be nice to move to a 
DirectByteBuffer compressed output stream. It should be quite viable since 
there are now compression methods for working over these directly, but that 
should probably be a follow up ticket. Do you want to file, or shall I?

bq. BufferedDataOutputStreamPlus.close() will clean the buffer even if it might 
not own the buffer such as when it is provided to the constructor. It also 
doesn't check if the channel is null.

We don't call close in situations where this is a problem

bq. When used from the commit log without a channel it will throw an NPE if it 
exceeds the capacity of the buffer when it goes to flush

We presize the buffer correctly

Still, on both these counts we could offer better safety if we wanted, without 
much cost. If we used a DataOutputBuffer it would solve these problems, and we 
could assert that the final buffer is == the provided buffer...

bq. It almost seems like there is a case for a version just for wrapping fixed 
size bytebuffers.

Possibly. I'm on the fence, since it's extra code cache and class hierarchy 
pollution, with limited positive impact. We would need to duplicate the whole 
of BufferedDataOutputStreamPlus. You could benchmark to see if there's an 
appreciable difference? If the difference is small (which I expect it is, given 
branch prediction), I would prefer to avoid the pollution.


was (Author: benedict):
bq. In most cases we opt to use the channel when available. ... Where we could 
really do better is when using compression

You're right. I thought we had more paths leftover, but it is just compression 
really. That's a much cleaner state of affairs!

It looks like compression will typically be under the stack buffer size, so we 
don't have such a problem, but it would still be nice to move to a 
DirectByteBuffer compressed output stream. It should be quite viable since 
there are now compression methods for working over these directly, but that 
should probably be a follow up ticket. Do you want to file, or shall I?

bq. BufferedDataOutputStreamPlus.close() will clean the buffer even if it might 
not own the buffer such as when it is provided to the constructor. It also 
doesn't check if the channel is null.

We don't call close in situations where this is a problem

bq. When used from the commit log without a channel it will throw an NPE if it 
exceeds the capacity of the buffer when it goes to flush

We presize the buffer correctly

Still, on both these counts we could offer better safety if we wanted, without 
much cost. If we used a DataOutputBuffer it would solve these problems, and we 
could assert that the final buffer is == the provided buffer...

bq. It almost seems like there is a case for a version just for wrapping fixed 
size bytebuffers.

Possibly. I'm on the fence, since it's extra code cache and class hierarchy 
pollution, with limited positive impact. We would need to duplicate the whole 
of BufferedDataOutputStreamPlus. You could benchmark to see if there's an 
appreciable difference? If the difference is small, I would prefer to avoid the 
pollution.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Ariel Weisberg (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390570#comment-14390570
 ] 

Ariel Weisberg commented on CASSANDRA-8670:
---

CASSANDRA-8887 would be part of it. Maybe an umbrella ticket and do that first 
to figure out how hook the compression libraries up. Can you file?

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9085) Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread T Jake Luciani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

T Jake Luciani resolved CASSANDRA-9085.
---
Resolution: Fixed

 Bind JMX to localhost unless explicitly configured otherwise
 

 Key: CASSANDRA-9085
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9085
 Project: Cassandra
  Issue Type: Bug
Reporter: T Jake Luciani
Assignee: T Jake Luciani
Priority: Critical
 Fix For: 2.1.4, 2.0.14


 Cassandra's default JMX config can lead to someone executing arbitrary code:  
 see http://www.mail-archive.com/user@cassandra.apache.org/msg41819.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread jake 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1.4-rel [created] c041ea8b3


Bind JMX to localhost unless explicitly configured otherwise


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c041ea8b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c041ea8b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c041ea8b

Branch: refs/heads/cassandra-2.1.4-rel
Commit: c041ea8b3748089937168839791a6d64382b34de
Parents: 7cc1cf0
Author: T Jake Luciani j...@apache.org
Authored: Wed Mar 25 12:27:50 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Mon Mar 30 09:38:50 2015 -0400

--
 CHANGES.txt |  3 +
 NEWS.txt|  6 ++
 bin/cassandra.bat   | 11 ++-
 build.xml   |  3 +-
 conf/cassandra-env.ps1  | 13 +++-
 conf/cassandra-env.sh   | 21 --
 debian/changelog|  6 ++
 .../cassandra/service/CassandraDaemon.java  | 78 ++--
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 9 files changed, 157 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 954fa3f..50c7967 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.1.4
+ * If no JMX flags are set start a localhost only JMX service
+
 2.1.3
  * Fix HSHA/offheap_objects corruption (CASSANDRA-8719)
  * Upgrade libthrift to 0.9.2 (CASSANDRA-8685)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 602770c..076885c 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,12 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+2.1.4
+=
+
+The default JMX config now listens to localhost only. You must enable 
+the other JMX flags in cassandra-env.sh manually.
+
 2.1.3
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/bin/cassandra.bat
--
diff --git a/bin/cassandra.bat b/bin/cassandra.bat
index 99b291a..fefd3fc 100644
--- a/bin/cassandra.bat
+++ b/bin/cassandra.bat
@@ -65,10 +65,13 @@ set JAVA_OPTS=-ea^
  -XX:MaxTenuringThreshold=1^
  -XX:CMSInitiatingOccupancyFraction=75^
  -XX:+UseCMSInitiatingOccupancyOnly^
- -Dcom.sun.management.jmxremote.port=7199^
- -Dcom.sun.management.jmxremote.ssl=false^
- -Dcom.sun.management.jmxremote.authenticate=false^
- -Dlogback.configurationFile=logback.xml
+ -Dlogback.configurationFile=logback.xml^
+ -Dcassandra.jmx.local.port=7199
+REM  JMX REMOTE ACCESS SETTINGS SEE: 
https://wiki.apache.org/cassandra/JmxSecurity ***
+REM -Dcom.sun.management.jmxremote.port=7199^
+REM -Dcom.sun.management.jmxremote.ssl=false^
+REM -Dcom.sun.management.jmxremote.authenticate=true^
+REM -Dcom.sun.management.jmxremote.password.file=C:\jmxremote.password
 
 REM * CLASSPATH library setting *
 REM Ensure that any user defined CLASSPATH variables are not used on startup

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/build.xml
--
diff --git a/build.xml b/build.xml
index eaef534..cf401e4 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.1.3/
+property name=base.version value=2.1.4/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/
@@ -1113,6 +1113,7 @@
 jvmarg value=-Xss256k/
 jvmarg 
value=-Dcassandra.memtable_row_overhead_computation_step=100/
 jvmarg 
value=-Dcassandra.test.use_prepared=${cassandra.test.use_prepared}/
+   jvmarg value=-Dcassandra.jmx.local.port=7199 /
jvmarg value=-Dcassandra.test.offsetseed=@{poffset}/
optjvmargs/
 classpath

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/conf/cassandra-env.ps1
--
diff --git a/conf/cassandra-env.ps1 b/conf/cassandra-env.ps1
index 9c6b6f4..7a71a13 100644
---

[2/2] cassandra git commit: Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread jake 
Bind JMX to localhost unless explicitly configured otherwise


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2433068c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2433068c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2433068c

Branch: refs/heads/cassandra-2.0.14-rel
Commit: 2433068c157cd21d4217c98ec5c0b9a2c07e288d
Parents: 20593c2
Author: T Jake Luciani j...@apache.org
Authored: Wed Mar 25 12:04:09 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Mon Mar 30 09:40:38 2015 -0400

--
 CHANGES.txt |  3 +
 NEWS.txt|  5 ++
 bin/cassandra.bat   | 11 ++-
 build.xml   |  5 +-
 conf/cassandra-env.sh   | 22 --
 debian/changelog|  6 ++
 .../cassandra/service/CassandraDaemon.java  | 79 +++-
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 8 files changed, 150 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f98bb3f..7a3a4ea 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.0.14:
+ * If no JMX flags are set start a localhost only JMX service
+
 2.0.13:
  * Add offline tool to relevel sstables (CASSANDRA-8301)
  * Preserve stream ID for more protocol errors (CASSANDRA-8848)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 5a4bc3d..82f643f 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,11 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+2.0.14
+==
+The default JMX config now listens to localhost only. You must enable 
+the other JMX flags in cassandra-env.sh manually.
+
 2.0.13
 ==
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/bin/cassandra.bat
--
diff --git a/bin/cassandra.bat b/bin/cassandra.bat
index 8a3bf7f..c316d7f 100644
--- a/bin/cassandra.bat
+++ b/bin/cassandra.bat
@@ -41,11 +41,14 @@ set JAVA_OPTS=-ea^
  -XX:MaxTenuringThreshold=1^
  -XX:CMSInitiatingOccupancyFraction=75^
  -XX:+UseCMSInitiatingOccupancyOnly^
- -Dcom.sun.management.jmxremote.port=7199^
- -Dcom.sun.management.jmxremote.ssl=false^
- -Dcom.sun.management.jmxremote.authenticate=false^
  -Dlog4j.configuration=log4j-server.properties^
- -Dlog4j.defaultInitOverride=true
+ -Dlog4j.defaultInitOverride=true^
+ -Dcassandra.jmx.local.port=7199
+REM  JMX REMOTE ACCESS SETTINGS SEE: 
https://wiki.apache.org/cassandra/JmxSecurity ***
+REM -Dcom.sun.management.jmxremote.port=7199^
+REM -Dcom.sun.management.jmxremote.ssl=false^
+REM -Dcom.sun.management.jmxremote.authenticate=true^
+REM -Dcom.sun.management.jmxremote.password.file=C:\jmxremote.password
 
 REM * CLASSPATH library setting *
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/build.xml
--
diff --git a/build.xml b/build.xml
index bdbd10a..e644b71 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.0.13/
+property name=base.version value=2.0.14/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/
@@ -1035,7 +1035,8 @@
 jvmarg value=-Dlog4j.configuration=log4j-junit.properties /
 jvmarg value=-Djava.awt.headless=true/
 jvmarg value=-javaagent:${basedir}/lib/jamm-0.2.5.jar /
-jvmarg value=-ea/
+jvmarg value=-Dcassandra.jmx.local.port=7199 /
+   jvmarg value=-ea/
 jvmarg value=-Xss256k/
 optjvmargs/
 classpath

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index dfe8184..70d86cf 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -261,9 +261,21 @@ JVM_OPTS=$JVM_OPTS -Djava.net.preferIPv4Stack=true
 #

[1/2] cassandra git commit: bump versions

2015-04-01 Thread jake 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0.14-rel [created] 2433068c1


bump versions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/20593c24
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/20593c24
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/20593c24

Branch: refs/heads/cassandra-2.0.14-rel
Commit: 20593c24a9f8c07aff8f848f4b5774ed88ea4687
Parents: a936d7e
Author: T Jake Luciani j...@apache.org
Authored: Thu Mar 12 21:01:11 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Mar 12 21:01:11 2015 -0400

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/20593c24/build.xml
--
diff --git a/build.xml b/build.xml
index 78c6db3..bdbd10a 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.0.12/
+property name=base.version value=2.0.13/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/20593c24/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 53fa20f..2c80800 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.0.13) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani j...@apache.org  Thu, 12 Mar 2015 21:00:06 -0400
+
 cassandra (2.0.12) unstable; urgency=medium
 
   * New release

[jira] [Commented] (CASSANDRA-8672) Ambiguous WriteTimeoutException while completing pending CAS commits

2015-04-01 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390545#comment-14390545
 ] 

Sylvain Lebresne commented on CASSANDRA-8672:
-

bq. What is the reason of having `WriteType.CAS` if I have to retry using 
serial reads for both CAS and SIMPLE timeouts?

Where in my answer did you see me suggesting that your *have to* retry using 
serial reads for timeouts with {{WriteType.SIMPLE}}?

 Ambiguous WriteTimeoutException while completing pending CAS commits
 

 Key: CASSANDRA-8672
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8672
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Stefan Podkowinski
Assignee: Tyler Hobbs
Priority: Minor
  Labels: CAS
 Fix For: 3.0


 Any CAS update has a chance to trigger a pending/stalled commit of any 
 previously agreed on CAS update. After completing the pending commit, the CAS 
 operation will resume to execute the actual update and also possibly create a 
 new commit. See StorageProxy.cas()
 Theres two possbile execution paths that might end up throwing a 
 WriteTimeoutException:
 cas() - beginAndRepairPaxos() - commitPaxos()
 cas() - commitPaxos()
 Unfortunatelly clients catching a WriteTimeoutException won't be able to tell 
 at which stage the commit failed. My guess would be that most developers are 
 not aware that the beginAndRepairPaxos() could also trigger a write and 
 assume that write timeouts would refer to a timeout while writting the actual 
 CAS update. Its therefor not safe to assume that successive CAS or SERIAL 
 read operations will cause a (write-)timeouted CAS operation to get 
 eventually applied. Although some [best-practices 
 advise|http://www.datastax.com/dev/blog/cassandra-error-handling-done-right] 
 claims otherwise.
 At this point the safest bet is possibly to retry the complete business 
 transaction in case of an WriteTimeoutException. However, as theres a chance 
 that the timeout occurred while writing the actual CAS operation, another 
 write could potentially complete it and our CAS condition will get a 
 different result upon retry.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Ariel Weisberg (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-8670:
--
Attachment: OutputStreamBench.java

I updated the microbenchmark to do what I think is the right think WRT to dead 
code elimination and constant folding. Results don't change, but I it looks 
more like I know what we I am doing.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9085) Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread T Jake Luciani (JIRA) 
T Jake Luciani created CASSANDRA-9085:
-

 Summary: Bind JMX to localhost unless explicitly configured 
otherwise
 Key: CASSANDRA-9085
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9085
 Project: Cassandra
  Issue Type: Bug
Reporter: T Jake Luciani
Assignee: T Jake Luciani
Priority: Critical
 Fix For: 2.1.4, 2.0.14


Cassandra's default JMX config can lead to someone executing arbitrary code:  
see http://www.mail-archive.com/user@cassandra.apache.org/msg41819.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Ariel Weisberg (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390570#comment-14390570
 ] 

Ariel Weisberg edited comment on CASSANDRA-8670 at 4/1/15 1:54 PM:
---

CASSANDRA-8887 would be part of it. Maybe an umbrella ticket and do that first 
to figure out how hook the compression libraries up. Can you file?

Also on trying to understand code cache pollution. How much do you really know 
about how many instructions are emitted when the JVM can inline and duplicate 
stuff out the yin yang?


was (Author: aweisberg):
CASSANDRA-8887 would be part of it. Maybe an umbrella ticket and do that first 
to figure out how hook the compression libraries up. Can you file?

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Git Push Summary

2015-04-01 Thread jake 
Repository: cassandra
Updated Tags:  refs/tags/cassandra-2.1.4 [created] 86ae5d077

Git Push Summary

2015-04-01 Thread jake 
Repository: cassandra
Updated Tags:  refs/tags/cassandra-2.0.14 [created] 513be774e

[jira] [Commented] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows

2015-04-01 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390519#comment-14390519
 ] 

Sylvain Lebresne commented on CASSANDRA-8979:
-

I'm not sure unconditionally skipping the top-level partition tombstone if it's 
live is a good idea, because that means that if you do a repair with nodes both 
prior and after this patch, all partitions will mismatch (they will include the 
live partition tombstone before this patch, not after it). And that's probably 
not something we should do in a minor upgrade.

Now, I don't think including the top-level partition tombstone is really a 
problem in general, it's only when the partition ends up having no live cells 
(when the partition is empty) that we should skip it so it's equivalent to 
having no partition at all. If we only skip it in that case, that will avoid 
the repair hell in mixed version cluster.

 MerkleTree mismatch for deleted and non-existing rows
 -

 Key: CASSANDRA-8979
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8979
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Stefan Podkowinski
Assignee: Stefan Podkowinski
 Fix For: 2.1.4, 2.0.14

 Attachments: 8979-AvoidBufferAllocation-2.0_patch.txt, 
 cassandra-2.0-8979-lazyrow_patch.txt, cassandra-2.0-8979-validator_patch.txt, 
 cassandra-2.0-8979-validatortest_patch.txt, 
 cassandra-2.1-8979-lazyrow_patch.txt, cassandra-2.1-8979-validator_patch.txt


 Validation compaction will currently create different hashes for rows that 
 have been deleted compared to nodes that have not seen the rows at all or 
 have already compacted them away. 
 In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to 
 prevent hashing of expired tombstones. This still seems to be in place, but 
 does not address the issue completely. Or there was a change in 2.0 that 
 rendered the patch ineffective. 
 The problem is that rowHash() in the Validator will return a new hash in any 
 case, whether the PrecompactedRow did actually update the digest or not. This 
 will lead to the case that a purged, PrecompactedRow will not change the 
 digest, but we end up with a different tree compared to not having rowHash 
 called at all (such as in case the row already doesn't exist).
 As an implication, repair jobs will constantly detect mismatches between 
 older sstables containing purgable rows and nodes that have already compacted 
 these rows. After transfering the reported ranges, the newly created sstables 
 will immediately get deleted again during the following compaction. This will 
 happen for each repair run over again until the sstable with the purgable row 
 finally gets compacted. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Ariel Weisberg (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390541#comment-14390541
 ] 

Ariel Weisberg commented on CASSANDRA-8670:
---

In what scenarios do we end up actually writing to the native methods right 
now? In most cases we opt to use the channel when available. The only time we 
should be using WrappedDataOutputStreamPlus is when we aren't actually writing 
to the channel of a file or socket.

Where we could really do better is when using compression, we could have the 
compression wrap a direct buffer which wraps the channel and avoid relying on 
the built in mechanisms for getting data off heap.

I had some other refrigerator moments overnight as well.

BufferedDataOutputStreamPlus.close() will clean the buffer even if it might not 
own the buffer such as when it is provided to the constructor. It also doesn't 
check if the channel is null.

When used from the commit log without a channel it will throw an NPE if it 
exceeds the capacity of the buffer when it goes to flush. I suppose one runtime 
exception is as good as another. There is also the extra bounds check in 
ensureRemaining() which always seemed a little useless to me and the fact that 
it will not drop through and do efficient copies for direct byte buffers. It 
almost seems like there is a case for a version just for wrapping fixed size 
bytebuffers. 

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r1670648 - in /cassandra/site: publish/download/index.html publish/index.html src/settings.py

2015-04-01 Thread jake 
Author: jake
Date: Wed Apr  1 13:42:28 2015
New Revision: 1670648

URL: http://svn.apache.org/r1670648
Log:
version bumps

Modified:
cassandra/site/publish/download/index.html
cassandra/site/publish/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/download/index.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1670648r1=1670647r2=1670648view=diff
==
--- cassandra/site/publish/download/index.html (original)
+++ cassandra/site/publish/download/index.html Wed Apr  1 13:42:28 2015
@@ -55,21 +55,21 @@
bThere are currently two active releases available:/b
br/
p
-  The latest release of Apache Cassandra is 2.1.3
-  (released on 2015-02-17).  iIf you're just
+  The latest release of Apache Cassandra is 2.1.4
+  (released on 2015-04-01).  iIf you're just
   starting out and not yet in production, download this one./i
 /p
   
ul
  li
a class=filename 
- 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.3/apache-cassandra-2.1.3-bin.tar.gz;
+ 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz;
  onclick=javascript: 
pageTracker._trackPageview('/clicks/binary_download');
-apache-cassandra-2.1.3-bin.tar.gz
+apache-cassandra-2.1.4-bin.tar.gz
/a
-   [a 
href=http://www.apache.org/dist/cassandra/2.1.3/apache-cassandra-2.1.3-bin.tar.gz.asc;PGP/a]
-   [a 
href=http://www.apache.org/dist/cassandra/2.1.3/apache-cassandra-2.1.3-bin.tar.gz.md5;MD5/a]
-   [a 
href=http://www.apache.org/dist/cassandra/2.1.3/apache-cassandra-2.1.3-bin.tar.gz.sha1;SHA1/a]
+   [a 
href=http://www.apache.org/dist/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.asc;PGP/a]
+   [a 
href=http://www.apache.org/dist/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.md5;MD5/a]
+   [a 
href=http://www.apache.org/dist/cassandra/2.1.4/apache-cassandra-2.1.4-bin.tar.gz.sha1;SHA1/a]
  /li
  li
a href=http://wiki.apache.org/cassandra/DebianPackaging;Debian 
installation instructions/a
@@ -77,16 +77,16 @@
/ul
 
p
- The bmost stable/b release of Apache Cassandra is 2.0.13
- (released on 2015-03-16).  iIf you are in production or planning to be 
soon, download this one./i
+ The bmost stable/b release of Apache Cassandra is 2.0.14
+ (released on 2015-04-01).  iIf you are in production or planning to be 
soon, download this one./i
/p
 
ul
  li
-   a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.13/apache-cassandra-2.0.13-bin.tar.gz;apache-cassandra-2.0.13-bin.tar.gz/a
-   [a 
href=http://www.apache.org/dist/cassandra/2.0.13/apache-cassandra-2.0.13-bin.tar.gz.asc;PGP/a]
-   [a 
href=http://www.apache.org/dist/cassandra/2.0.13/apache-cassandra-2.0.13-bin.tar.gz.md5;MD5/a]
-   [a 
href=http://www.apache.org/dist/cassandra/2.0.13/apache-cassandra-2.0.13-bin.tar.gz.sha1;SHA1/a]
+   a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz;apache-cassandra-2.0.14-bin.tar.gz/a
+   [a 
href=http://www.apache.org/dist/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.asc;PGP/a]
+   [a 
href=http://www.apache.org/dist/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.md5;MD5/a]
+   [a 
href=http://www.apache.org/dist/cassandra/2.0.14/apache-cassandra-2.0.14-bin.tar.gz.sha1;SHA1/a]
  /li
 li
 a href=http://wiki.apache.org/cassandra/DebianPackaging;Debian 
installation instructions/a
@@ -155,20 +155,20 @@
   ul
 li
 a class=filename 
-   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.3/apache-cassandra-2.1.3-src.tar.gz;
+   
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz;
onclick=javascript: 
pageTracker._trackPageview('/clicks/source_download');
-  apache-cassandra-2.1.3-src.tar.gz
+  apache-cassandra-2.1.4-src.tar.gz
 /a
-[a 
href=http://www.apache.org/dist/cassandra/2.1.3/apache-cassandra-2.1.3-src.tar.gz.asc;PGP/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.1.3/apache-cassandra-2.1.3-src.tar.gz.md5;MD5/a]
-[a 
href=http://www.apache.org/dist/cassandra/2.1.3/apache-cassandra-2.1.3-src.tar.gz.sha1;SHA1/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.asc;PGP/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.md5;MD5/a]
+[a 
href=http://www.apache.org/dist/cassandra/2.1.4/apache-cassandra-2.1.4-src.tar.gz.sha1;SHA1/a]
 /li
   
 li
-a class=filename 
href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.0.13/apache-cassandra-2.0.13-src.tar.gz;apache-cassandra-2.0.13-src.tar.gz/a
-[a

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Ariel Weisberg (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390595#comment-14390595
 ] 

Ariel Weisberg commented on CASSANDRA-8670:
---

On more thinking I don't think having close clean the buffer if we allow people 
to supply one is great. It could be double free or use after free if someone 
makes a mistake.

I think we should had least have FileUtil.clean null out the pointer to the 
buffer before/after cleaning (whichever is possible) so we get as immediate a 
failure as possible. Duplicates or slices of the buffer won't pick up that the 
pointer was nulled, but it's better then nothing. We should also assert that 
the buffer wasn't provided in the constructor and throw an exception if someone 
did that and then called close.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390598#comment-14390598
 ] 

Benedict commented on CASSANDRA-8670:
-

bq. Also on trying to understand code cache pollution. How much do you really 
know about how many instructions are emitted when the JVM can inline and 
duplicate stuff out the yin yang?

Well, there are a lot of heuristics to apply, that are admittedly limited and 
imperfect. But in general: hotspot won't inline megamorphic call sites, and 
even bimorphic callsites are unlikely to be (i think probably never) inlined, 
only given a static despatch fast path. These heuristics are enough to guide 
decisions around this sufficiently well in my experience. If there are multiple 
implementations viable at any moment, then the callsite will not be inlined, at 
most its location will be, and even if it _is_ inlined, this doesn't 
necessarily pollute the code cache, since inlined methods are small (as are 
most methods) and adjacent occupancy of the cache is essentially free, unless 
the overall method size exceeds the cache line boundary.

In general my view, the simplest heuristic is: if the benefit is small, and it 
increases the number of active call sites, then let's not. This works from a 
code management as well as a cache pollution perspective at once.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9085) Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390588#comment-14390588
 ] 

T Jake Luciani commented on CASSANDRA-9085:
---

2.0.x branch diff 
https://github.com/apache/cassandra/compare/cassandra-2.0.13...tjake:cassandra-2.0.14-build
2.1.x branch diff 
https://github.com/apache/cassandra/compare/cassandra-2.1.3...tjake:cassandra-2.1.4-build

 Bind JMX to localhost unless explicitly configured otherwise
 

 Key: CASSANDRA-9085
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9085
 Project: Cassandra
  Issue Type: Bug
Reporter: T Jake Luciani
Assignee: T Jake Luciani
Priority: Critical
 Fix For: 2.1.4, 2.0.14


 Cassandra's default JMX config can lead to someone executing arbitrary code:  
 see http://www.mail-archive.com/user@cassandra.apache.org/msg41819.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Ariel Weisberg (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390595#comment-14390595
 ] 

Ariel Weisberg edited comment on CASSANDRA-8670 at 4/1/15 2:15 PM:
---

On more thinking I don't think having close clean the buffer if we allow people 
to supply one is great. It could be double free or use after free if someone 
makes a mistake.

-I think we should had least have FileUtil.clean null out the pointer to the 
buffer before/after cleaning (whichever is possible) so we get as immediate a 
failure as possible.- (This is done by the cleaner) Duplicates or slices of the 
buffer won't pick up that the pointer was nulled, but it's better then nothing. 
We should also assert that the buffer wasn't provided in the constructor and 
throw an exception if someone did that and then called close.


was (Author: aweisberg):
On more thinking I don't think having close clean the buffer if we allow people 
to supply one is great. It could be double free or use after free if someone 
makes a mistake.

I think we should had least have FileUtil.clean null out the pointer to the 
buffer before/after cleaning (whichever is possible) so we get as immediate a 
failure as possible. Duplicates or slices of the buffer won't pick up that the 
pointer was nulled, but it's better then nothing. We should also assert that 
the buffer wasn't provided in the constructor and throw an exception if someone 
did that and then called close.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Ariel Weisberg (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390595#comment-14390595
 ] 

Ariel Weisberg edited comment on CASSANDRA-8670 at 4/1/15 2:16 PM:
---

On more thinking I don't think having close clean the buffer if we allow people 
to supply one is great. It could be double free or use after free if someone 
makes a mistake.

-I think we should had least have FileUtil.clean null out the pointer to the 
buffer before/after cleaning (whichever is possible) so we get as immediate a 
failure as possible.- -(This is done by the cleaner)- Except not all buffers we 
create will have cleaners... Duplicates or slices of the buffer won't pick up 
that the pointer was nulled, but it's better then nothing. We should also 
assert that the buffer wasn't provided in the constructor and throw an 
exception if someone did that and then called close.


was (Author: aweisberg):
On more thinking I don't think having close clean the buffer if we allow people 
to supply one is great. It could be double free or use after free if someone 
makes a mistake.

-I think we should had least have FileUtil.clean null out the pointer to the 
buffer before/after cleaning (whichever is possible) so we get as immediate a 
failure as possible.- (This is done by the cleaner) Duplicates or slices of the 
buffer won't pick up that the pointer was nulled, but it's better then nothing. 
We should also assert that the buffer wasn't provided in the constructor and 
throw an exception if someone did that and then called close.

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Roger Schildmeijer (JIRA) 
Roger Schildmeijer created CASSANDRA-9086:
-

 Summary: CAS precondition mismatch - row is still inserted
 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Attachments: conditional_insert_failure_cleaned.txt

Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};

CQL statement: 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Roger Schildmeijer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roger Schildmeijer updated CASSANDRA-9086:
--
Description: 
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
{quote}
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{quote}

CQL statement:
{quote} 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
{quote}

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702


  was:
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
{quote}
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{quote}

CQL statement: 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702



 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {quote}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {quote}
 CQL statement:
 {quote} 
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {quote}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Roger Schildmeijer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roger Schildmeijer updated CASSANDRA-9086:
--
Description: 
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
{quote}
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{quote}

CQL statement: 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702


  was:
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};

CQL statement: 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702



 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {quote}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {quote}
 CQL statement: 
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Roger Schildmeijer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roger Schildmeijer updated CASSANDRA-9086:
--
Description: 
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

{code:xml}
We are having a quite simple cf schema:
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{code}

CQL statement:
{quote} 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
{quote}

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702


  was:
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
{quote}
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{quote}

CQL statement:
{quote} 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
{quote}

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702



 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 {code:xml}
 We are having a quite simple cf schema:
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {code}
 CQL statement:
 {quote} 
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {quote}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8887) Direct (de)compression of internode communication

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390606#comment-14390606
 ] 

Benedict commented on CASSANDRA-8887:
-

I'm confused by the description of this ticket: how does this affect GC load 
(beyond any other long term objects)? At least since LZ4 was made the internode 
compression algorithm, there is no extra allocation on the decompression path 
(it looks like Snappy was wasteful, so in 2.0 this might be true, but I doubt 
we're going to be backporting this to 2.0)

There is still a benefit to be had, but I think we should consider replacing 
our compression scheme wholesale, so that the compression algorithm is more 
easily swapped in, like we have for files.

 Direct (de)compression of internode communication
 -

 Key: CASSANDRA-8887
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8887
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Matt Stump
Assignee: Ariel Weisberg

 Internode compression is on by default. Currently we allocate one set of 
 buffers for the raw data, and then compress which results in another set of 
 buffers. This greatly increases the GC load. We can decrease the GC load by 
 doing direct compression/decompression of the communication buffers. This is 
 the same work as done in CASSANDRA-8464 but applied to internode 
 communication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Roger Schildmeijer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roger Schildmeijer updated CASSANDRA-9086:
--
Description: 
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
{code:xml}
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{code}

CQL statement:
{quote} 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
{quote}

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702


  was:
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

{code:xml}
We are having a quite simple cf schema:
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{code}

CQL statement:
{quote} 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
{quote}

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702



 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {code:xml}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {code}
 CQL statement:
 {quote} 
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {quote}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Roger Schildmeijer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roger Schildmeijer updated CASSANDRA-9086:
--
Description: 
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
{code:xml}
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{code}

CQL statement:
{code:xml}
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
{code}

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702


  was:
Cassandra claims CAS precondition didn't match current values[1], but the row 
is still inserted.

We are having a quite simple cf schema:
{code:xml}
create table index (
  u text PRIMARY KEY,
  abc setuuid,
) WITH
  compaction={'class': 'LeveledCompactionStrategy'};
{code}

CQL statement:
{quote} 
INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
{quote}

Sometimes, especially during some write load, Cassandra claims the conditional 
insert wasn't satisfied ([applied] false), but the row is still inserted.

The response from the conditional insert contains [applied]=false + the data we 
tried to insert.

A full CQL trace is attached.


[1] CAS precondition  CAS precondition 
org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not match 
current values ColumnFamily(username_index -{deletedAt=-9223372036854775808, 
localDeletion=2147483647, ranges=[abc-abc:!, deletedAt=1427889576525999, 
localDeletion=1427889564]}- 
[:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
 | 11:59:37.060 | /10.0.0.1 | 533702



 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {code:xml}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {code}
 CQL statement:
 {code:xml}
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {code}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8276) Unusable prepared statement with 65k parameters

2015-04-01 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390615#comment-14390615
 ] 

Benjamin Lerer commented on CASSANDRA-8276:
---

I tested the problem on the latest 2.0 and 2.1 and faced a different issue than 
what you described. The test ran forever but I never received an {{UNPREPARED}} 
answer.

The first problem that I found was that the unit test seems to be wrong. Prior 
to 3.0 an {{IN}} restriction containing multiple time the same value  will 
return the same number of time the same result. In the case of your unit test 
you should receive 65535 times the row: ('0', 0). 

If you replace the 65535 zero by 65535 different values the test will work 
fine. 

The problem comes from the fact that specifing the same value for the partition 
key in the {{IN}} restriction a number of time greater than the page size 
breaks the paging. Cassandra keeps on returning the same page of data forever.

The only way I have found to fix the problem will be to add some extra data in 
the {{ClientState}} but this would be a breaking change.

As the use case is a really specific one and as the problem will be fixed in 
3.0, I would be in favor of marking this ticket as {{won't fix}}.
[~thobbs] what is your opinion?

  

 Unusable prepared statement with 65k parameters
 ---

 Key: CASSANDRA-8276
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8276
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 2.0.10
 Java driver 2.0.8-SNAPSHOT
Reporter: Pierre Laporte
Assignee: Benjamin Lerer
 Fix For: 2.1.5


 We had an issue 
 ([JAVA-515|https://datastax-oss.atlassian.net/browse/JAVA-515]) in the 
 java-driver when the number of parameters in a statement is greater than the 
 supported limit (65k).
 I added a limit-test to verify that prepared statements with 65535 parameters 
 were accepted by the driver, but ran into an issue on the Cassandra side.
 Basically, the test runs forever, because the driver receives an inconsistent 
 answer from Cassandra.  When we prepare the statement, C* answers that it is 
 correctly prepared, however when we try to execute it, we receive a 
 {{UNPREPARED}} answer.
 [Here is the 
 code|https://github.com/datastax/java-driver/blob/JAVA-515/driver-core/src/test/java/com/datastax/driver/core/PreparedStatementTest.java#L448]
  to reproduce the issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9087) Error in snappy-1.0.5 on REHL 5

2015-04-01 Thread dewnie perera (JIRA) 
dewnie perera created CASSANDRA-9087:


 Summary: Error in snappy-1.0.5 on REHL 5
 Key: CASSANDRA-9087
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9087
 Project: Cassandra
  Issue Type: Bug
 Environment: apache-cassandra-2.1.3
 RHEL5
Reporter: dewnie perera


I'm getting a below exception when running cassandra on RHEL5. 

java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:322)
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
at org.xerial.snappy.Snappy.clinit(Snappy.java:48)
at 
org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:47)
at 
org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:57)
at 
org.apache.cassandra.io.compress.SnappyCompressor.clinit(SnappyCompressor.java:39)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:191)
at 
org.apache.cassandra.io.compress.CompressionParameters.parseCompressorClass(CompressionParameters.java:162)
at 
org.apache.cassandra.io.compress.CompressionParameters.init(CompressionParameters.java:73)
at 
org.apache.cassandra.io.compress.CompressionParameters.create(CompressionParameters.java:66)
at 
org.apache.cassandra.config.CFMetaData.internalFromThrift(CFMetaData.java:1087)
at 
org.apache.cassandra.config.CFMetaData.fromThrift(CFMetaData.java:994)
at 
org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:1540)
at 
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4402)
at 
org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4386)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at 
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsatisfiedLinkError: 
/tmp/snappy-1.0.5.2-libsnappyjava.so: /usr/lib64/libstdc++.so.6: version 
`GLIBCXX_3.4.9' not found (required by /tmp/snappy-1.0.5.2-libsnappyjava.so)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
at java.lang.Runtime.load0(Runtime.java:795)
at java.lang.System.load(System.java:1062)
at org.xerial.snappy.SnappyNativeLoader.load(SnappyNativeLoader.java:39)
... 26 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-04-01 Thread Benedict (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390630#comment-14390630
 ] 

Benedict commented on CASSANDRA-8670:
-

bq. We should also assert that the buffer wasn't provided in the constructor 
and throw an exception if someone did that and then called close.

If we forbid this entirely, and only expose it via DataOutputBuffer, then the 
close method is harmless (as it's a no-op)

 Large columns + NIO memory pooling causes excessive direct memory usage
 ---

 Key: CASSANDRA-8670
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
 Fix For: 3.0

 Attachments: OutputStreamBench.java, largecolumn_test.py


 If you provide a large byte array to NIO and ask it to populate the byte 
 array from a socket it will allocate a thread local byte buffer that is the 
 size of the requested read no matter how large it is. Old IO wraps new IO for 
 sockets (but not files) so old IO is effected as well.
 Even If you are using Buffered{Input | Output}Stream you can end up passing a 
 large byte array to NIO. The byte array read method will pass the array to 
 NIO directly if it is larger than the internal buffer.  
 Passing large cells between nodes as part of intra-cluster messaging can 
 cause the NIO pooled buffers to quickly reach a high watermark and stay 
 there. This ends up costing 2x the largest cell size because there is a 
 buffer for input and output since they are different threads. This is further 
 multiplied by the number of nodes in the cluster - 1 since each has a 
 dedicated thread pair with separate thread locals.
 Anecdotally it appears that the cost is doubled beyond that although it isn't 
 clear why. Possibly the control connections or possibly there is some way in 
 which multiple 
 Need a workload in CI that tests the advertised limits of cells on a cluster. 
 It would be reasonable to ratchet down the max direct memory for the test to 
 trigger failures if a memory pooling issue is introduced. I don't think we 
 need to test concurrently pulling in a lot of them, but it should at least 
 work serially.
 The obvious fix to address this issue would be to read in smaller chunks when 
 dealing with large values. I think small should still be relatively large (4 
 megabytes) so that code that is reading from a disk can amortize the cost of 
 a seek. It can be hard to tell what the underlying thing being read from is 
 going to be in some of the contexts where we might choose to implement 
 switching to reading chunks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread jake 
Bind JMX to localhost unless explicitly configured otherwise


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2433068c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2433068c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2433068c

Branch: refs/heads/cassandra-2.0
Commit: 2433068c157cd21d4217c98ec5c0b9a2c07e288d
Parents: 20593c2
Author: T Jake Luciani j...@apache.org
Authored: Wed Mar 25 12:04:09 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Mon Mar 30 09:40:38 2015 -0400

--
 CHANGES.txt |  3 +
 NEWS.txt|  5 ++
 bin/cassandra.bat   | 11 ++-
 build.xml   |  5 +-
 conf/cassandra-env.sh   | 22 --
 debian/changelog|  6 ++
 .../cassandra/service/CassandraDaemon.java  | 79 +++-
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 8 files changed, 150 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f98bb3f..7a3a4ea 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.0.14:
+ * If no JMX flags are set start a localhost only JMX service
+
 2.0.13:
  * Add offline tool to relevel sstables (CASSANDRA-8301)
  * Preserve stream ID for more protocol errors (CASSANDRA-8848)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 5a4bc3d..82f643f 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,11 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+2.0.14
+==
+The default JMX config now listens to localhost only. You must enable 
+the other JMX flags in cassandra-env.sh manually.
+
 2.0.13
 ==
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/bin/cassandra.bat
--
diff --git a/bin/cassandra.bat b/bin/cassandra.bat
index 8a3bf7f..c316d7f 100644
--- a/bin/cassandra.bat
+++ b/bin/cassandra.bat
@@ -41,11 +41,14 @@ set JAVA_OPTS=-ea^
  -XX:MaxTenuringThreshold=1^
  -XX:CMSInitiatingOccupancyFraction=75^
  -XX:+UseCMSInitiatingOccupancyOnly^
- -Dcom.sun.management.jmxremote.port=7199^
- -Dcom.sun.management.jmxremote.ssl=false^
- -Dcom.sun.management.jmxremote.authenticate=false^
  -Dlog4j.configuration=log4j-server.properties^
- -Dlog4j.defaultInitOverride=true
+ -Dlog4j.defaultInitOverride=true^
+ -Dcassandra.jmx.local.port=7199
+REM  JMX REMOTE ACCESS SETTINGS SEE: 
https://wiki.apache.org/cassandra/JmxSecurity ***
+REM -Dcom.sun.management.jmxremote.port=7199^
+REM -Dcom.sun.management.jmxremote.ssl=false^
+REM -Dcom.sun.management.jmxremote.authenticate=true^
+REM -Dcom.sun.management.jmxremote.password.file=C:\jmxremote.password
 
 REM * CLASSPATH library setting *
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/build.xml
--
diff --git a/build.xml b/build.xml
index bdbd10a..e644b71 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.0.13/
+property name=base.version value=2.0.14/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/
@@ -1035,7 +1035,8 @@
 jvmarg value=-Dlog4j.configuration=log4j-junit.properties /
 jvmarg value=-Djava.awt.headless=true/
 jvmarg value=-javaagent:${basedir}/lib/jamm-0.2.5.jar /
-jvmarg value=-ea/
+jvmarg value=-Dcassandra.jmx.local.port=7199 /
+   jvmarg value=-ea/
 jvmarg value=-Xss256k/
 optjvmargs/
 classpath

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index dfe8184..70d86cf 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -261,9 +261,21 @@ JVM_OPTS=$JVM_OPTS -Djava.net.preferIPv4Stack=true
 #

[1/3] cassandra git commit: bump versions

2015-04-01 Thread jake 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 2cf4ca3a9 - 38a04eab6


bump versions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/20593c24
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/20593c24
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/20593c24

Branch: refs/heads/cassandra-2.0
Commit: 20593c24a9f8c07aff8f848f4b5774ed88ea4687
Parents: a936d7e
Author: T Jake Luciani j...@apache.org
Authored: Thu Mar 12 21:01:11 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Mar 12 21:01:11 2015 -0400

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/20593c24/build.xml
--
diff --git a/build.xml b/build.xml
index 78c6db3..bdbd10a 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.0.12/
+property name=base.version value=2.0.13/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/20593c24/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 53fa20f..2c80800 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.0.13) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani j...@apache.org  Thu, 12 Mar 2015 21:00:06 -0400
+
 cassandra (2.0.12) unstable; urgency=medium
 
   * New release

cassandra git commit: fixup

2015-04-01 Thread benedict 
Repository: cassandra
Updated Branches:
  refs/heads/trunk 82a526863 - e617d8461


fixup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e617d846
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e617d846
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e617d846

Branch: refs/heads/trunk
Commit: e617d84616893c974e6132f9dab7748645466632
Parents: 82a5268
Author: Benedict Elliott Smith bened...@apache.org
Authored: Wed Apr 1 15:55:02 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Wed Apr 1 15:55:02 2015 +0100

--
 .../org/apache/cassandra/locator/NetworkTopologyStrategy.java  | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e617d846/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
--
diff --git a/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java 
b/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
index fb291b5..6959f29 100644
--- a/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
+++ b/src/java/org/apache/cassandra/locator/NetworkTopologyStrategy.java
@@ -85,7 +85,7 @@ public class NetworkTopologyStrategy extends 
AbstractReplicationStrategy
 // replicas we have found in each DC
 MapString, SetInetAddress dcReplicas = new 
HashMap(datacenters.size());
 for (Map.EntryString, Integer dc : datacenters.entrySet())
-dcReplicas.put(dc.getKey(), new HashSet(dc.getValue()));
+dcReplicas.put(dc.getKey(), new 
HashSetInetAddress(dc.getValue()));
 
 Topology topology = tokenMetadata.getTopology();
 // all endpoints in each DC, so we can check when we have exhausted 
all the members of a DC
@@ -97,13 +97,13 @@ public class NetworkTopologyStrategy extends 
AbstractReplicationStrategy
 // tracks the racks we have already placed replicas in
 MapString, SetString seenRacks = new HashMap(datacenters.size());
 for (Map.EntryString, Integer dc : datacenters.entrySet())
-seenRacks.put(dc.getKey(), new HashSet());
+seenRacks.put(dc.getKey(), new HashSetString());
 
 // tracks the endpoints that we skipped over while looking for unique 
racks
 // when we relax the rack uniqueness we can append this to the current 
result so we don't have to wind back the iterator
 MapString, SetInetAddress skippedDcEndpoints = new 
HashMap(datacenters.size());
 for (Map.EntryString, Integer dc : datacenters.entrySet())
-skippedDcEndpoints.put(dc.getKey(), new LinkedHashSet());
+skippedDcEndpoints.put(dc.getKey(), new 
LinkedHashSetInetAddress());
 
 IteratorToken tokenIter = 
TokenMetadata.ringIterator(tokenMetadata.sortedTokens(), searchToken, false);
 while (tokenIter.hasNext()  !hasSufficientReplicas(dcReplicas, 
allEndpoints))

[4/6] cassandra git commit: Merge branch 'cassandra-2.0.14-build' into cassandra-2.0

2015-04-01 Thread jake 
Merge branch 'cassandra-2.0.14-build' into cassandra-2.0

Conflicts:
CHANGES.txt
build.xml
debian/changelog


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/38a04eab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/38a04eab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/38a04eab

Branch: refs/heads/cassandra-2.1
Commit: 38a04eab6a18fe81e91cd0b7ecf00c34c443f2b5
Parents: 2cf4ca3 2433068
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 10:45:51 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 10:45:51 2015 -0400

--
 CHANGES.txt |  4 +-
 NEWS.txt|  5 ++
 bin/cassandra.bat   | 11 ++-
 build.xml   |  2 +-
 conf/cassandra-env.sh   | 22 --
 debian/changelog|  9 +++
 .../cassandra/service/CassandraDaemon.java  | 79 +++-
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 8 files changed, 151 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/38a04eab/CHANGES.txt
--
diff --cc CHANGES.txt
index b6b5caf,7a3a4ea..fe1aa37
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,39 -1,5 +1,41 @@@
--2.0.14:
 - * If no JMX flags are set start a localhost only JMX service
++2.0.15:
 + * Avoid race in cancelling compactions (CASSANDRA-9070)
 + * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
 + * Don't set clientMode to true when bulk-loading sstables to avoid
 +   a NullPointerException (CASSANDRA-8238)
 + * Fix ignored index_interval change in ALTER TABLE statements 
(CASSANDRA-7976)
 + * Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
 + * java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
 + * disk full when running cleanup (CASSANDRA-9036)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Lower logging level from ERROR to DEBUG when a scheduled schema pull
 +   cannot be completed due to a node being down (CASSANDRA-9032)
 + * Fix MOVED_NODE client event (CASSANDRA-8516)
 + * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
 + * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * (cqlsh) Allow increasing CSV field size limit through
 +   cqlshrc config option (CASSANDRA-8934)
 + * Stop logging range tombstones when exceeding the threshold
 +   (CASSANDRA-8559)
 + * Fix NullPointerException when nodetool getendpoints is run
 +   against invalid keyspaces or tables (CASSANDRA-8950)
 + * Allow specifying the tmp dir (CASSANDRA-7712)
 + * Improve compaction estimated tasks estimation (CASSANDRA-8904)
 + * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
 + * Expose commit log archive status via JMX (CASSANDRA-8734)
 + * Provide better exceptions for invalid replication strategy parameters
 +   (CASSANDRA-8909)
 + * Fix regression in mixed single and multi-column relation support for
 +   SELECT statements (CASSANDRA-8613)
 + * Add ability to limit number of native connections (CASSANDRA-8086)
 + * Fix CQLSSTableWriter throwing exception and spawning threads
 +   (CASSANDRA-8808)
 + * Fix MT mismatch between empty and GC-able data (CASSANDRA-8979)
 + * Fix incorrect validation when snapshotting single table (CASSANDRA-8056)
 +
++2.0.14
++ * Bind JMX to localhost unless explicitly configured otherwise 
(CASSANDRA-9085)
  
  2.0.13:
   * Add offline tool to relevel sstables (CASSANDRA-8301)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/38a04eab/build.xml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/38a04eab/debian/changelog
--
diff --cc debian/changelog
index 2c80800,257d636..cd183a3
--- a/debian/changelog
+++ b/debian/changelog
@@@ -1,3 -1,9 +1,12 @@@
++ HEAD
++===
+ cassandra (2.0.14) unstable; urgency=medium
+ 
+   * New release
+ 
+  -- Jake Luciani j...@apache.org  Thu, 26 Mar 2015 14:57:39 -0400
+ 
++ cassandra-2.0.14-build
  cassandra (2.0.13) unstable; urgency=medium
  
* New release

[1/6] cassandra git commit: bump versions

2015-04-01 Thread jake 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 730cc6064 - 150f82736


bump versions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/20593c24
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/20593c24
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/20593c24

Branch: refs/heads/cassandra-2.1
Commit: 20593c24a9f8c07aff8f848f4b5774ed88ea4687
Parents: a936d7e
Author: T Jake Luciani j...@apache.org
Authored: Thu Mar 12 21:01:11 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Thu Mar 12 21:01:11 2015 -0400

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/20593c24/build.xml
--
diff --git a/build.xml b/build.xml
index 78c6db3..bdbd10a 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.0.12/
+property name=base.version value=2.0.13/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/20593c24/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 53fa20f..2c80800 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.0.13) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani j...@apache.org  Thu, 12 Mar 2015 21:00:06 -0400
+
 cassandra (2.0.12) unstable; urgency=medium
 
   * New release

[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2015-04-01 Thread jake 
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/beb5ffd0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/beb5ffd0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/beb5ffd0

Branch: refs/heads/cassandra-2.1
Commit: beb5ffd0785c1be2564a5e1f09b2907f573d83d2
Parents: 730cc60 38a04ea
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 10:49:53 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 10:49:53 2015 -0400

--

--

[jira] [Comment Edited] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Philip Thompson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390760#comment-14390760
 ] 

Philip Thompson edited comment on CASSANDRA-9086 at 4/1/15 3:09 PM:


Are you able to reproduce this consistently? Do you know if the row existed or 
not beforehand? It will help us know if this is incorrectly overwriting data, 
or should be applying, but incorrectly reporting a failure.

What is the RF on the keyspace holding this table?


was (Author: philipthompson):
Are you able to reproduce this consistently? Do you know if the row existed or 
not beforehand? It will help us know if this is incorrectly overwriting data, 
or should be applying, but incorrectly reporting a failure.

 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Fix For: 2.0.15

 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {code:xml}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {code}
 CQL statement:
 {code:xml}
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {code}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Philip Thompson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390760#comment-14390760
 ] 

Philip Thompson commented on CASSANDRA-9086:


Are you able to reproduce this consistently? Do you know if the row existed or 
not beforehand? It will help us know if this is incorrectly overwriting data, 
or should be applying, but incorrectly reporting a failure.

 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Fix For: 2.0.15

 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {code:xml}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {code}
 CQL statement:
 {code:xml}
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {code}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8672) Ambiguous WriteTimeoutException while completing pending CAS commits

2015-04-01 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390761#comment-14390761
 ] 

Sylvain Lebresne commented on CASSANDRA-8672:
-

I don't understand your arguments. You seems to suggest that you'd be content 
with a new prepare flag, but it wouldn't change when we use 
{{WriteType.SIMPLE}}. All I'm saying is that adding a new {{CAS_PREPARE}} is 
only about splitting the {{CAS}} case in two and I don't understand why you're 
saying that distinguishing CAS and SIMPLE is pointless while it would become 
somehow great with a CAS_PREPARE.

 Ambiguous WriteTimeoutException while completing pending CAS commits
 

 Key: CASSANDRA-8672
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8672
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Stefan Podkowinski
Assignee: Tyler Hobbs
Priority: Minor
  Labels: CAS
 Fix For: 3.0


 Any CAS update has a chance to trigger a pending/stalled commit of any 
 previously agreed on CAS update. After completing the pending commit, the CAS 
 operation will resume to execute the actual update and also possibly create a 
 new commit. See StorageProxy.cas()
 Theres two possbile execution paths that might end up throwing a 
 WriteTimeoutException:
 cas() - beginAndRepairPaxos() - commitPaxos()
 cas() - commitPaxos()
 Unfortunatelly clients catching a WriteTimeoutException won't be able to tell 
 at which stage the commit failed. My guess would be that most developers are 
 not aware that the beginAndRepairPaxos() could also trigger a write and 
 assume that write timeouts would refer to a timeout while writting the actual 
 CAS update. Its therefor not safe to assume that successive CAS or SERIAL 
 read operations will cause a (write-)timeouted CAS operation to get 
 eventually applied. Although some [best-practices 
 advise|http://www.datastax.com/dev/blog/cassandra-error-handling-done-right] 
 claims otherwise.
 At this point the safest bet is possibly to retry the complete business 
 transaction in case of an WriteTimeoutException. However, as theres a chance 
 that the timeout occurred while writing the actual CAS operation, another 
 write could potentially complete it and our CAS condition will get a 
 different result upon retry.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8035) 2.0.x repair causes large increasein client latency even for small datasets

2015-04-01 Thread Philip Thompson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8035:
---
Fix Version/s: 2.0.15

 2.0.x repair causes large increasein client latency even for small datasets
 ---

 Key: CASSANDRA-8035
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8035
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: c-2.0.10, 3 nodes per @ DCs.  Load  50 MB
Reporter: Chris Burroughs
 Fix For: 2.0.15

 Attachments: cl-latency.png, cpu-idle.png, keyspace-99p.png, 
 row-cache-hit-rate.png


 Running repair causes a significnat increase in client latency even when the 
 total amount of data per node is very small.
 Each node 900 req/s and during normal operations the 99p Client Request 
 Lantecy is less than 4 ms and usually less than 1ms.  During repair the 
 latency increases to within 4-10ms on all nodes.  I am unable to find any 
 resource based explantion for this.  Several graphs are attached to 
 summarize.  Repair started at about 10:10 and finished around 10:25.
  * Client Request Latency goes up significantly.
  * Local keyspace read latency is flat.  I interpret this to mean that it's 
 purly coordinator overhead that's causing the slowdown.
  * Row cache hit rate is unaffected ( and is very high).  Between these two 
 metrics I don't think there is any doubt that virtually all reads are being 
 satisfied in memory.
  * There is plenty of available cpu.  Aggregate cpu used (mostly nic) did go 
 up during this.
 Having more/larger keyspaces seems to make it worse.  Having two keyspaces on 
 this cluster (still with total size  RAM) caused larger increases in 
 latency which would have made for better graphs but it pushed the cluster 
 well outsid of SLAs and we needed to move the second keyspace.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-9088) Don't include tmp files in SSTableOfflineRelevel

2015-04-01 Thread Marcus Eriksson (JIRA) 
Marcus Eriksson created CASSANDRA-9088:
--

 Summary: Don't include tmp files in SSTableOfflineRelevel
 Key: CASSANDRA-9088
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9088
 Project: Cassandra
  Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
 Fix For: 2.0.15
 Attachments: 0001-dont-include-tmp-files-in-offline-relevel.patch

Avoid including tmp files in offline relevel



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Philip Thompson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390848#comment-14390848
 ] 

Philip Thompson commented on CASSANDRA-9086:


rf=3 per dc in NTS? 

 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Fix For: 2.0.15

 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {code:xml}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {code}
 CQL statement:
 {code:xml}
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {code}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f

2015-04-01 Thread Joseph Chu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Chu updated CASSANDRA-9083:
--
Comment: was deleted

(was: I'm using Python 2.7.3)

 cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
 ---

 Key: CASSANDRA-9083
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 2.1.3
Reporter: Joseph Chu
Assignee: Tyler Hobbs
Priority: Minor
  Labels: cqlsh
 Fix For: 2.1.5


 Executing a COPY command from an external file using the cqlsh -f or the 
 SOURCE command results in the error:
 filename.cql:7:descriptor 'lower' requires a 'str' object but received a 
 'unicode'
 Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which 
 makes use of codecs.open() instead of open(), which returns a unicode object. 
 The offending line of code that returns the error seems to be in cqlsh, line 
 1415:
 copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f

2015-04-01 Thread Joseph Chu (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390886#comment-14390886
 ] 

Joseph Chu commented on CASSANDRA-9083:
---

I'm using Python 2.7.3

 cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
 ---

 Key: CASSANDRA-9083
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 2.1.3
Reporter: Joseph Chu
Assignee: Tyler Hobbs
Priority: Minor
  Labels: cqlsh
 Fix For: 2.1.5


 Executing a COPY command from an external file using the cqlsh -f or the 
 SOURCE command results in the error:
 filename.cql:7:descriptor 'lower' requires a 'str' object but received a 
 'unicode'
 Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which 
 makes use of codecs.open() instead of open(), which returns a unicode object. 
 The offending line of code that returns the error seems to be in cqlsh, line 
 1415:
 copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f

2015-04-01 Thread Joseph Chu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Chu updated CASSANDRA-9083:
--
Comment: was deleted

(was: I'm using Python 2.7.3)

 cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
 ---

 Key: CASSANDRA-9083
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 2.1.3
Reporter: Joseph Chu
Assignee: Tyler Hobbs
Priority: Minor
  Labels: cqlsh
 Fix For: 2.1.5


 Executing a COPY command from an external file using the cqlsh -f or the 
 SOURCE command results in the error:
 filename.cql:7:descriptor 'lower' requires a 'str' object but received a 
 'unicode'
 Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which 
 makes use of codecs.open() instead of open(), which returns a unicode object. 
 The offending line of code that returns the error seems to be in cqlsh, line 
 1415:
 copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-2.0.14-build' into cassandra-2.0

2015-04-01 Thread jake 
Merge branch 'cassandra-2.0.14-build' into cassandra-2.0

Conflicts:
CHANGES.txt
build.xml
debian/changelog


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/38a04eab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/38a04eab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/38a04eab

Branch: refs/heads/cassandra-2.0
Commit: 38a04eab6a18fe81e91cd0b7ecf00c34c443f2b5
Parents: 2cf4ca3 2433068
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 10:45:51 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 10:45:51 2015 -0400

--
 CHANGES.txt |  4 +-
 NEWS.txt|  5 ++
 bin/cassandra.bat   | 11 ++-
 build.xml   |  2 +-
 conf/cassandra-env.sh   | 22 --
 debian/changelog|  9 +++
 .../cassandra/service/CassandraDaemon.java  | 79 +++-
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 8 files changed, 151 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/38a04eab/CHANGES.txt
--
diff --cc CHANGES.txt
index b6b5caf,7a3a4ea..fe1aa37
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,39 -1,5 +1,41 @@@
--2.0.14:
 - * If no JMX flags are set start a localhost only JMX service
++2.0.15:
 + * Avoid race in cancelling compactions (CASSANDRA-9070)
 + * More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
 + * Don't set clientMode to true when bulk-loading sstables to avoid
 +   a NullPointerException (CASSANDRA-8238)
 + * Fix ignored index_interval change in ALTER TABLE statements 
(CASSANDRA-7976)
 + * Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
 + * java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
 + * disk full when running cleanup (CASSANDRA-9036)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Lower logging level from ERROR to DEBUG when a scheduled schema pull
 +   cannot be completed due to a node being down (CASSANDRA-9032)
 + * Fix MOVED_NODE client event (CASSANDRA-8516)
 + * Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
 + * Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * (cqlsh) Allow increasing CSV field size limit through
 +   cqlshrc config option (CASSANDRA-8934)
 + * Stop logging range tombstones when exceeding the threshold
 +   (CASSANDRA-8559)
 + * Fix NullPointerException when nodetool getendpoints is run
 +   against invalid keyspaces or tables (CASSANDRA-8950)
 + * Allow specifying the tmp dir (CASSANDRA-7712)
 + * Improve compaction estimated tasks estimation (CASSANDRA-8904)
 + * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
 + * Expose commit log archive status via JMX (CASSANDRA-8734)
 + * Provide better exceptions for invalid replication strategy parameters
 +   (CASSANDRA-8909)
 + * Fix regression in mixed single and multi-column relation support for
 +   SELECT statements (CASSANDRA-8613)
 + * Add ability to limit number of native connections (CASSANDRA-8086)
 + * Fix CQLSSTableWriter throwing exception and spawning threads
 +   (CASSANDRA-8808)
 + * Fix MT mismatch between empty and GC-able data (CASSANDRA-8979)
 + * Fix incorrect validation when snapshotting single table (CASSANDRA-8056)
 +
++2.0.14
++ * Bind JMX to localhost unless explicitly configured otherwise 
(CASSANDRA-9085)
  
  2.0.13:
   * Add offline tool to relevel sstables (CASSANDRA-8301)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/38a04eab/build.xml
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/38a04eab/debian/changelog
--
diff --cc debian/changelog
index 2c80800,257d636..cd183a3
--- a/debian/changelog
+++ b/debian/changelog
@@@ -1,3 -1,9 +1,12 @@@
++ HEAD
++===
+ cassandra (2.0.14) unstable; urgency=medium
+ 
+   * New release
+ 
+  -- Jake Luciani j...@apache.org  Thu, 26 Mar 2015 14:57:39 -0400
+ 
++ cassandra-2.0.14-build
  cassandra (2.0.13) unstable; urgency=medium
  
* New release

[5/7] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2015-04-01 Thread jake 
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/beb5ffd0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/beb5ffd0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/beb5ffd0

Branch: refs/heads/trunk
Commit: beb5ffd0785c1be2564a5e1f09b2907f573d83d2
Parents: 730cc60 38a04ea
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 10:49:53 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 10:49:53 2015 -0400

--

--

[jira] [Commented] (CASSANDRA-7724) Native-Transport threads get stuck in StorageProxy.preparePaxos with no one making progress

2015-04-01 Thread Philip Thompson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390829#comment-14390829
 ] 

Philip Thompson commented on CASSANDRA-7724:


Have you reproduced on 2.1.3 or a newer 2.0 version? 

 Native-Transport threads get stuck in StorageProxy.preparePaxos with no one 
 making progress
 ---

 Key: CASSANDRA-7724
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7724
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Linux 3.13.11-4 #4 SMP PREEMPT x86_64 Intel(R) Core(TM) 
 i7 CPU 950 @ 3.07GHz GenuineIntel 
 java version 1.8.0_05
 Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
 Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)
 cassandra 2.0.9
Reporter: Anton Lebedevich
 Attachments: aggregateddump.txt, cassandra.threads2


 We've got a lot of write timeouts (cas) when running 
 INSERT INTO cas_demo(pri_id, sec_id, flag, something) VALUES(?, ?, ?, ?) IF 
 NOT EXISTS
  from 16 connections in parallel using the same pri_id and different sec_id.
 Doing the same from 4 connections in parallel works ok.
 All configuration values are at their default values.
 CREATE TABLE cas_demo (
   pri_id varchar,
   sec_id varchar,
   flag boolean,
   something setvarchar,
   PRIMARY KEY (pri_id, sec_id)
 );
 CREATE INDEX cas_demo_flag ON cas_demo(flag);
 Full thread dump is attached. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9086) CAS precondition mismatch - row is still inserted

2015-04-01 Thread Roger Schildmeijer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390852#comment-14390852
 ] 

Roger Schildmeijer commented on CASSANDRA-9086:
---

I think I see some patterns in the cql traces (the one attached + other) when 
this happens.
1, Preparing ballot happens at least twice
2, Finishing incomplete paxos round Commit(..) happens 

I also noticed that 
org.apache.cassandra.metrics:type=ClientRequest,scope=CASWrite,name=ConditionNotMet
 and
org.apache.cassandra.metrics:type=ClientRequest,scope=CASWrite,name=UnfinishedCommit
 are ~equal on all my nodes


 CAS precondition mismatch - row is still inserted
 -

 Key: CASSANDRA-9086
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9086
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra version 2.0.11
 24 nodes in 4 datacenters (6 nodes per dc)
Reporter: Roger Schildmeijer
Priority: Critical
 Fix For: 2.0.15

 Attachments: conditional_insert_failure_cleaned.txt


 Cassandra claims CAS precondition didn't match current values[1], but the row 
 is still inserted.
 We are having a quite simple cf schema:
 {code:xml}
 create table index (
   u text PRIMARY KEY,
   abc setuuid,
 ) WITH
   compaction={'class': 'LeveledCompactionStrategy'};
 {code}
 CQL statement:
 {code:xml}
 INSERT INTO index(u, abc) VALUES(?, ?) IF NOT EXISTS
 {code}
 Sometimes, especially during some write load, Cassandra claims the 
 conditional insert wasn't satisfied ([applied] false), but the row is still 
 inserted.
 The response from the conditional insert contains [applied]=false + the data 
 we tried to insert.
 A full CQL trace is attached.
 [1] CAS precondition  CAS precondition 
 org.apache.cassandra.cql3.statements.CQL3CasConditions@1b2576ce does not 
 match current values ColumnFamily(username_index 
 -{deletedAt=-9223372036854775808, localDeletion=2147483647, 
 ranges=[abc-abc:!, deletedAt=1427889576525999, localDeletion=1427889564]}- 
 [:false:0@1427889576526000,abc:39f1a470ee694761a8197c0d0ada8e8f:false:0@1427889576526000,])
  | 11:59:37.060 | /10.0.0.1 | 533702



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9077) Deleting an element from a List which is null throws a NPE

2015-04-01 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390867#comment-14390867
 ] 

Benjamin Lerer commented on CASSANDRA-9077:
---

+1

 Deleting an element from a List which is null throws a NPE
 --

 Key: CASSANDRA-9077
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9077
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: dan jatnieks
Assignee: Jeff Jirsa
Priority: Minor
 Attachments: 9077-stacktrace.log, CASSANDRA-9077-v3.txt, 
 cassandra-9077-option1-v2.txt, cassandra-9077-option1.txt, 
 cassandra-9077-option2.txt


 I am seeing an NPE on the latest 2.1 branch with this sequence of deletes 
 from a list - first delete the entire list, then attempt to delete one 
 element.
 I expected to see {{List index 0 out of bound, list has size 0}} but instead 
 got an NPE.
 {noformat}
 ./bin/cqlsh
 Connected to Test Cluster at 127.0.0.1:9042.
 [cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
 Use HELP for help.
 cqlsh use frozen_collections ;
 cqlsh:frozen_collections DROP TABLE IF EXISTS t;
 cqlsh:frozen_collections CREATE TABLE t (id text PRIMARY KEY, l listtext, 
 s settext);
 cqlsh:frozen_collections INSERT INTO t (id, l, s) VALUES ('user', ['1'], 
 {'1'});
 cqlsh:frozen_collections
 cqlsh:frozen_collections DELETE l FROM t WHERE id ='user';
 cqlsh:frozen_collections //INSERT INTO t (id, l) VALUES ('user', ['1']);
 cqlsh:frozen_collections DELETE l[0] FROM t WHERE id = 'user';
 ServerError: ErrorMessage code= [Server error] 
 message=java.lang.NullPointerException
 cqlsh:frozen_collections
 cqlsh:frozen_collections DELETE s FROM t WHERE id ='user';
 cqlsh:frozen_collections DELETE s['1'] FROM t WHERE id = 'user';
 {noformat}
 It appears the {{DELETE emails...}} directly followed by {{DELETE 
 emails[0]...}} is the offending sequence. Either one alone works fine, as 
 does adding an intervening insert/update.
 The same sequence performed on a Set rather than List works (as shown above).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9087) Error in snappy-1.0.5 on REHL 5

2015-04-01 Thread Michael Shuler (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390869#comment-14390869
 ] 

Michael Shuler commented on CASSANDRA-9087:
---

{noformat}
$ ./switch_snappy
This script will enable either 1.0.4 or 1.0.5 snappy jar files. This
is needed because Snappy 1.0.5 will not run on old version of Linux
due to clib mismach issues.

Please specify what version of snappy to set up:
1.0.4
1.0.5
{noformat}

RHEL5 and the error you got is exactly why this little script exists. You 
should be able to find switch_snappy at {{/usr/share/cassandra/switch_snappy}}. 
Stop cassandra, run it with {{1.0.4}} as an argument, and then start cassandra 
back up.

 Error in snappy-1.0.5 on REHL 5
 ---

 Key: CASSANDRA-9087
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9087
 Project: Cassandra
  Issue Type: Bug
 Environment: apache-cassandra-2.1.3
  RHEL5
Reporter: dewnie perera
 Fix For: 2.1.5


 I'm getting a below exception when running cassandra on RHEL5. 
 
 {code}
 java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at 
 org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:322)
   at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
   at org.xerial.snappy.Snappy.clinit(Snappy.java:48)
   at 
 org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:47)
   at 
 org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:57)
   at 
 org.apache.cassandra.io.compress.SnappyCompressor.clinit(SnappyCompressor.java:39)
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:191)
   at 
 org.apache.cassandra.io.compress.CompressionParameters.parseCompressorClass(CompressionParameters.java:162)
   at 
 org.apache.cassandra.io.compress.CompressionParameters.init(CompressionParameters.java:73)
   at 
 org.apache.cassandra.io.compress.CompressionParameters.create(CompressionParameters.java:66)
   at 
 org.apache.cassandra.config.CFMetaData.internalFromThrift(CFMetaData.java:1087)
   at 
 org.apache.cassandra.config.CFMetaData.fromThrift(CFMetaData.java:994)
   at 
 org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:1540)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4402)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4386)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.UnsatisfiedLinkError: 
 /tmp/snappy-1.0.5.2-libsnappyjava.so: /usr/lib64/libstdc++.so.6: version 
 `GLIBCXX_3.4.9' not found (required by /tmp/snappy-1.0.5.2-libsnappyjava.so)
   at java.lang.ClassLoader$NativeLibrary.load(Native Method)
   at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
   at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
   at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
   at java.lang.Runtime.load0(Runtime.java:795)
   at java.lang.System.load(System.java:1062)
   at org.xerial.snappy.SnappyNativeLoader.load(SnappyNativeLoader.java:39)
   ... 26 more{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9087) Error in snappy-1.0.5 on REHL 5

2015-04-01 Thread Michael Shuler (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler resolved CASSANDRA-9087.
---
   Resolution: Not a Problem
Fix Version/s: (was: 2.1.5)

Closing as not a problem - do let us know if this didn't work out.

 Error in snappy-1.0.5 on REHL 5
 ---

 Key: CASSANDRA-9087
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9087
 Project: Cassandra
  Issue Type: Bug
 Environment: apache-cassandra-2.1.3
  RHEL5
Reporter: dewnie perera
Assignee: Michael Shuler

 I'm getting a below exception when running cassandra on RHEL5. 
 
 {code}
 java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at 
 org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:322)
   at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
   at org.xerial.snappy.Snappy.clinit(Snappy.java:48)
   at 
 org.apache.cassandra.io.compress.SnappyCompressor.create(SnappyCompressor.java:47)
   at 
 org.apache.cassandra.io.compress.SnappyCompressor.isAvailable(SnappyCompressor.java:57)
   at 
 org.apache.cassandra.io.compress.SnappyCompressor.clinit(SnappyCompressor.java:39)
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:191)
   at 
 org.apache.cassandra.io.compress.CompressionParameters.parseCompressorClass(CompressionParameters.java:162)
   at 
 org.apache.cassandra.io.compress.CompressionParameters.init(CompressionParameters.java:73)
   at 
 org.apache.cassandra.io.compress.CompressionParameters.create(CompressionParameters.java:66)
   at 
 org.apache.cassandra.config.CFMetaData.internalFromThrift(CFMetaData.java:1087)
   at 
 org.apache.cassandra.config.CFMetaData.fromThrift(CFMetaData.java:994)
   at 
 org.apache.cassandra.thrift.CassandraServer.system_add_column_family(CassandraServer.java:1540)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4402)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$system_add_column_family.getResult(Cassandra.java:4386)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:205)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.UnsatisfiedLinkError: 
 /tmp/snappy-1.0.5.2-libsnappyjava.so: /usr/lib64/libstdc++.so.6: version 
 `GLIBCXX_3.4.9' not found (required by /tmp/snappy-1.0.5.2-libsnappyjava.so)
   at java.lang.ClassLoader$NativeLibrary.load(Native Method)
   at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
   at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
   at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
   at java.lang.Runtime.load0(Runtime.java:795)
   at java.lang.System.load(System.java:1062)
   at org.xerial.snappy.SnappyNativeLoader.load(SnappyNativeLoader.java:39)
   ... 26 more{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8672) Ambiguous WriteTimeoutException while completing pending CAS commits

2015-04-01 Thread Stefan Podkowinski (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390745#comment-14390745
 ] 

Stefan Podkowinski commented on CASSANDRA-8672:
---

You'd have to read serial to get a chance to find out if your cas operation has 
been applied or not. And no, you didn't suggest you absolutely have to do that. 
But for the sake of the discussion, lets assume users will be interested to 
find out if they need to retry a cas operation in case the operation hasn't 
been applied at all. 

 Ambiguous WriteTimeoutException while completing pending CAS commits
 

 Key: CASSANDRA-8672
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8672
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Stefan Podkowinski
Assignee: Tyler Hobbs
Priority: Minor
  Labels: CAS
 Fix For: 3.0


 Any CAS update has a chance to trigger a pending/stalled commit of any 
 previously agreed on CAS update. After completing the pending commit, the CAS 
 operation will resume to execute the actual update and also possibly create a 
 new commit. See StorageProxy.cas()
 Theres two possbile execution paths that might end up throwing a 
 WriteTimeoutException:
 cas() - beginAndRepairPaxos() - commitPaxos()
 cas() - commitPaxos()
 Unfortunatelly clients catching a WriteTimeoutException won't be able to tell 
 at which stage the commit failed. My guess would be that most developers are 
 not aware that the beginAndRepairPaxos() could also trigger a write and 
 assume that write timeouts would refer to a timeout while writting the actual 
 CAS update. Its therefor not safe to assume that successive CAS or SERIAL 
 read operations will cause a (write-)timeouted CAS operation to get 
 eventually applied. Although some [best-practices 
 advise|http://www.datastax.com/dev/blog/cassandra-error-handling-done-right] 
 claims otherwise.
 At this point the safest bet is possibly to retry the complete business 
 transaction in case of an WriteTimeoutException. However, as theres a chance 
 that the timeout occurred while writing the actual CAS operation, another 
 write could potentially complete it and our CAS condition will get a 
 different result upon retry.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: fix import spacing

2015-04-01 Thread jake 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 150f82736 - 28af4fa04


fix import spacing


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28af4fa0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28af4fa0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28af4fa0

Branch: refs/heads/cassandra-2.1
Commit: 28af4fa04fc84e61bf35bb3761c01f51533b9b15
Parents: 150f827
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 11:13:18 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 11:13:18 2015 -0400

--
 src/java/org/apache/cassandra/service/CassandraDaemon.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/28af4fa0/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index 412a0bf..b527528 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -26,7 +26,7 @@ import java.net.UnknownHostException;
 import java.rmi.registry.LocateRegistry;
 import java.rmi.server.RMIServerSocketFactory;
 import java.util.*;
-import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeUnit;
 import javax.management.MBeanServer;
 import javax.management.ObjectName;
 import javax.management.StandardMBean;

[jira] [Updated] (CASSANDRA-8041) Utility sstablesplit should prevent users from running when C* is running

2015-04-01 Thread Philip Thompson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8041:
---
Assignee: Branimir Lambov

 Utility sstablesplit should prevent users from running when C* is running
 -

 Key: CASSANDRA-8041
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8041
 Project: Cassandra
  Issue Type: Bug
  Components: Documentation  website, Tools
Reporter: Erick Ramirez
Assignee: Branimir Lambov
Priority: Minor
 Fix For: 2.0.15


 The sstablesplit utility is designed for use when C* is offline, but there is 
 nothing stopping the user from running it on a live system. There are also no 
 warning messages alerting the user to this effect.
 The help information should also be updated to explicitly state that the 
 utility should only be used when C* is offline.
 Finally, this utility is not included in any of the documentation. Please 
 update accordingly. Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7970) JSON support for CQL

2015-04-01 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390820#comment-14390820
 ] 

Sylvain Lebresne commented on CASSANDRA-7970:
-

Alright, +1, thanks.

 JSON support for CQL
 

 Key: CASSANDRA-7970
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
  Labels: client-impacting, cql3.3, docs-impacting
 Fix For: 3.0

 Attachments: 7970-trunk-v1.txt, 7970-trunk-v2.txt


 JSON is popular enough that not supporting it is becoming a competitive 
 weakness.  We can add JSON support in a way that is compatible with our 
 performance goals by *mapping* JSON to an existing schema: one JSON documents 
 maps to one CQL row.
 Thus, it is NOT a goal to support schemaless documents, which is a misfeature 
 [1] [2] [3].  Rather, it is to allow a convenient way to easily turn a JSON 
 document from a service or a user into a CQL row, with all the validation 
 that entails.
 Since we are not looking to support schemaless documents, we will not be 
 adding a JSON data type (CASSANDRA-6833) a la postgresql.  Rather, we will 
 map the JSON to UDT, collections, and primitive CQL types.
 Here's how this might look:
 {code}
 CREATE TYPE address (
   street text,
   city text,
   zip_code int,
   phones settext
 );
 CREATE TABLE users (
   id uuid PRIMARY KEY,
   name text,
   addresses maptext, address
 );
 INSERT INTO users JSON
 {‘id’: 4b856557-7153,
‘name’: ‘jbellis’,
‘address’: {“home”: {“street”: “123 Cassandra Dr”,
 “city”: “Austin”,
 “zip_code”: 78747,
 “phones”: [2101234567]}}};
 SELECT JSON id, address FROM users;
 {code}
 (We would also want to_json and from_json functions to allow mapping a single 
 column's worth of data.  These would not require extra syntax.)
 [1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
 [2] https://blog.compose.io/schema-less-is-usually-a-lie/
 [3] http://dl.acm.org/citation.cfm?id=2481247



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8022) cqlsh hangs indefinitely within a Docker container connecting to itself with hostname

2015-04-01 Thread Philip Thompson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8022:
---
Reproduced In: 2.1.0
Fix Version/s: 2.1.5

 cqlsh hangs indefinitely within a Docker container connecting to itself with 
 hostname
 -

 Key: CASSANDRA-8022
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8022
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: Ubuntu 14.04, running Docker, run inside a Ubuntu 14.04 
 container.  
Reporter: Matthew O'Riordan
  Labels: cqlsh
 Fix For: 2.1.5


 I am unable to use the `cqlsh` tool within a Docker container running 
 Cassandra.  Previously I would use the Java  Thrift based `cqlsh` tool as 
 follows:
 ```
 cqlsh --username cassandra --password whatever $(hostname)
 ```
 When I run the `cqlsh` command after attaching to a running container (I use 
 LXC containerisation that allows attaching to a running container and running 
 a console), it simply hangs and never reports an error.  With the `--debug` 
 flag on, I get the following with:
 **cqlsh 4.1.1**
 ```
 $ cqlsh --debug --username cassandra --password obfuscated $(hostname) 
 Using CQL driver: module 'cql' from 
 '/usr/share/cassandra/lib/cql-internal-only-1.4.1.zip/cql-1.4.1/cql/__init__.py'
 Using thrift lib: module 'thrift' from 
 '/usr/share/cassandra/lib/thrift-python-internal-only-0.9.1.zip/thrift/__init__.py'
 ```
 It then hangs in this state indefinitely, I have no errors from `cqlsh` and 
 no errors in the Cassandra log.
 **cqlsh 5.0.1**
 ```
 $ cqlsh --debug --username cassandra --passwor obfuscated $(hostname) 
 Using CQL driver: module 'cassandra' from 
 '/usr/share/cassandra/lib/cassandra-driver-internal-only-2.1.0.post.zip/cassandra-driver-2.1.0.post/cassandra/__init__.py'
 ```
 It then also hangs in this state indefinitely, I have no errors from `cqlsh` 
 and no errors in the Cassandra log.
 What's interesting, and quite confusing is that:
 * I can telnet within the container as follows `telnet $(hostname) 9042` and 
 I get a socket.  When trying to issue some commands, I see Protocol errors in 
 the Cassandra log thus verifying that the port is indeed open on the host 
 that resolves from $(hostname)
 * If I `cqlsh` from another container or another host to the Cassandra 
 container it works just fine.
 * I have tried disabling authentication altogether and using the 
 AllowAllAuthenticator, and I experience the same problem.
 * `nodetool` works fine
 In the mean time, I am forced to `cqlsh` from another container as a 
 workaround.  Happy to try and do anything require to diagnose the cause of 
 this problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f

2015-04-01 Thread Joseph Chu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Chu updated CASSANDRA-9083:
--
Comment: was deleted

(was: I'm using Python 2.7.3)

 cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
 ---

 Key: CASSANDRA-9083
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 2.1.3
Reporter: Joseph Chu
Assignee: Tyler Hobbs
Priority: Minor
  Labels: cqlsh
 Fix For: 2.1.5


 Executing a COPY command from an external file using the cqlsh -f or the 
 SOURCE command results in the error:
 filename.cql:7:descriptor 'lower' requires a 'str' object but received a 
 'unicode'
 Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which 
 makes use of codecs.open() instead of open(), which returns a unicode object. 
 The offending line of code that returns the error seems to be in cqlsh, line 
 1415:
 copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f

2015-04-01 Thread Joseph Chu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Chu updated CASSANDRA-9083:
--
Comment: was deleted

(was: I'm using Python 2.7.3)

 cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
 ---

 Key: CASSANDRA-9083
 URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
 Environment: 2.1.3
Reporter: Joseph Chu
Assignee: Tyler Hobbs
Priority: Minor
  Labels: cqlsh
 Fix For: 2.1.5


 Executing a COPY command from an external file using the cqlsh -f or the 
 SOURCE command results in the error:
 filename.cql:7:descriptor 'lower' requires a 'str' object but received a 
 'unicode'
 Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which 
 makes use of codecs.open() instead of open(), which returns a unicode object. 
 The offending line of code that returns the error seems to be in cqlsh, line 
 1415:
 copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/6] cassandra git commit: Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread jake 
Bind JMX to localhost unless explicitly configured otherwise


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2433068c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2433068c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2433068c

Branch: refs/heads/cassandra-2.1
Commit: 2433068c157cd21d4217c98ec5c0b9a2c07e288d
Parents: 20593c2
Author: T Jake Luciani j...@apache.org
Authored: Wed Mar 25 12:04:09 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Mon Mar 30 09:40:38 2015 -0400

--
 CHANGES.txt |  3 +
 NEWS.txt|  5 ++
 bin/cassandra.bat   | 11 ++-
 build.xml   |  5 +-
 conf/cassandra-env.sh   | 22 --
 debian/changelog|  6 ++
 .../cassandra/service/CassandraDaemon.java  | 79 +++-
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 8 files changed, 150 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f98bb3f..7a3a4ea 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.0.14:
+ * If no JMX flags are set start a localhost only JMX service
+
 2.0.13:
  * Add offline tool to relevel sstables (CASSANDRA-8301)
  * Preserve stream ID for more protocol errors (CASSANDRA-8848)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 5a4bc3d..82f643f 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,11 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+2.0.14
+==
+The default JMX config now listens to localhost only. You must enable 
+the other JMX flags in cassandra-env.sh manually.
+
 2.0.13
 ==
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/bin/cassandra.bat
--
diff --git a/bin/cassandra.bat b/bin/cassandra.bat
index 8a3bf7f..c316d7f 100644
--- a/bin/cassandra.bat
+++ b/bin/cassandra.bat
@@ -41,11 +41,14 @@ set JAVA_OPTS=-ea^
  -XX:MaxTenuringThreshold=1^
  -XX:CMSInitiatingOccupancyFraction=75^
  -XX:+UseCMSInitiatingOccupancyOnly^
- -Dcom.sun.management.jmxremote.port=7199^
- -Dcom.sun.management.jmxremote.ssl=false^
- -Dcom.sun.management.jmxremote.authenticate=false^
  -Dlog4j.configuration=log4j-server.properties^
- -Dlog4j.defaultInitOverride=true
+ -Dlog4j.defaultInitOverride=true^
+ -Dcassandra.jmx.local.port=7199
+REM  JMX REMOTE ACCESS SETTINGS SEE: 
https://wiki.apache.org/cassandra/JmxSecurity ***
+REM -Dcom.sun.management.jmxremote.port=7199^
+REM -Dcom.sun.management.jmxremote.ssl=false^
+REM -Dcom.sun.management.jmxremote.authenticate=true^
+REM -Dcom.sun.management.jmxremote.password.file=C:\jmxremote.password
 
 REM * CLASSPATH library setting *
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/build.xml
--
diff --git a/build.xml b/build.xml
index bdbd10a..e644b71 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.0.13/
+property name=base.version value=2.0.14/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/
@@ -1035,7 +1035,8 @@
 jvmarg value=-Dlog4j.configuration=log4j-junit.properties /
 jvmarg value=-Djava.awt.headless=true/
 jvmarg value=-javaagent:${basedir}/lib/jamm-0.2.5.jar /
-jvmarg value=-ea/
+jvmarg value=-Dcassandra.jmx.local.port=7199 /
+   jvmarg value=-ea/
 jvmarg value=-Xss256k/
 optjvmargs/
 classpath

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2433068c/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index dfe8184..70d86cf 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -261,9 +261,21 @@ JVM_OPTS=$JVM_OPTS -Djava.net.preferIPv4Stack=true
 #

[2/6] cassandra git commit: Bind JMX to localhost unless explicitly configured otherwise

2015-04-01 Thread jake 
Bind JMX to localhost unless explicitly configured otherwise


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c041ea8b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c041ea8b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c041ea8b

Branch: refs/heads/cassandra-2.1
Commit: c041ea8b3748089937168839791a6d64382b34de
Parents: 7cc1cf0
Author: T Jake Luciani j...@apache.org
Authored: Wed Mar 25 12:27:50 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Mon Mar 30 09:38:50 2015 -0400

--
 CHANGES.txt |  3 +
 NEWS.txt|  6 ++
 bin/cassandra.bat   | 11 ++-
 build.xml   |  3 +-
 conf/cassandra-env.ps1  | 13 +++-
 conf/cassandra-env.sh   | 21 --
 debian/changelog|  6 ++
 .../cassandra/service/CassandraDaemon.java  | 78 ++--
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 9 files changed, 157 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 954fa3f..50c7967 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.1.4
+ * If no JMX flags are set start a localhost only JMX service
+
 2.1.3
  * Fix HSHA/offheap_objects corruption (CASSANDRA-8719)
  * Upgrade libthrift to 0.9.2 (CASSANDRA-8685)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 602770c..076885c 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,12 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+2.1.4
+=
+
+The default JMX config now listens to localhost only. You must enable 
+the other JMX flags in cassandra-env.sh manually.
+
 2.1.3
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/bin/cassandra.bat
--
diff --git a/bin/cassandra.bat b/bin/cassandra.bat
index 99b291a..fefd3fc 100644
--- a/bin/cassandra.bat
+++ b/bin/cassandra.bat
@@ -65,10 +65,13 @@ set JAVA_OPTS=-ea^
  -XX:MaxTenuringThreshold=1^
  -XX:CMSInitiatingOccupancyFraction=75^
  -XX:+UseCMSInitiatingOccupancyOnly^
- -Dcom.sun.management.jmxremote.port=7199^
- -Dcom.sun.management.jmxremote.ssl=false^
- -Dcom.sun.management.jmxremote.authenticate=false^
- -Dlogback.configurationFile=logback.xml
+ -Dlogback.configurationFile=logback.xml^
+ -Dcassandra.jmx.local.port=7199
+REM  JMX REMOTE ACCESS SETTINGS SEE: 
https://wiki.apache.org/cassandra/JmxSecurity ***
+REM -Dcom.sun.management.jmxremote.port=7199^
+REM -Dcom.sun.management.jmxremote.ssl=false^
+REM -Dcom.sun.management.jmxremote.authenticate=true^
+REM -Dcom.sun.management.jmxremote.password.file=C:\jmxremote.password
 
 REM * CLASSPATH library setting *
 REM Ensure that any user defined CLASSPATH variables are not used on startup

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/build.xml
--
diff --git a/build.xml b/build.xml
index eaef534..cf401e4 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.1.3/
+property name=base.version value=2.1.4/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/
@@ -1113,6 +1113,7 @@
 jvmarg value=-Xss256k/
 jvmarg 
value=-Dcassandra.memtable_row_overhead_computation_step=100/
 jvmarg 
value=-Dcassandra.test.use_prepared=${cassandra.test.use_prepared}/
+   jvmarg value=-Dcassandra.jmx.local.port=7199 /
jvmarg value=-Dcassandra.test.offsetseed=@{poffset}/
optjvmargs/
 classpath

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c041ea8b/conf/cassandra-env.ps1
--
diff --git a/conf/cassandra-env.ps1 b/conf/cassandra-env.ps1
index 9c6b6f4..7a71a13 100644
--- a/conf/cassandra-env.ps1
+++ b/conf/cassandra-env.ps1
@@ -400,10 +400,17 @@ Function SetCassandraEnvironment
 #

[6/6] cassandra git commit: Merge branch 'cassandra-2.1.4-build' into cassandra-2.1

2015-04-01 Thread jake 
Merge branch 'cassandra-2.1.4-build' into cassandra-2.1

Conflicts:
CHANGES.txt
NEWS.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/150f8273
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/150f8273
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/150f8273

Branch: refs/heads/cassandra-2.1
Commit: 150f8273645e733f125870f58c03aa33b9395855
Parents: beb5ffd c041ea8
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 10:56:05 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 10:56:05 2015 -0400

--
 CHANGES.txt |  8 +-
 NEWS.txt|  7 +-
 bin/cassandra.bat   | 11 ++-
 build.xml   |  3 +-
 conf/cassandra-env.ps1  | 13 +++-
 conf/cassandra-env.sh   | 21 --
 debian/changelog|  6 ++
 .../cassandra/service/CassandraDaemon.java  | 78 ++--
 .../utils/RMIServerSocketFactoryImpl.java   | 34 +
 9 files changed, 157 insertions(+), 24 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/150f8273/CHANGES.txt
--
diff --cc CHANGES.txt
index 3b99c61,50c7967..913b570
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,119 -1,5 +1,121 @@@
- 2.1.4
++2.1.5
 + * Buffer bloom filter serialization (CASSANDRA-9066)
 + * Fix anti-compaction target bloom filter size (CASSANDRA-9060)
 + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047)
 + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034)
 + * Avoid overwriting index summaries for sstables with an older format that
 +   does not support downsampling; rebuild summaries on startup when this
 +   is detected (CASSANDRA-8993)
 + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
 + * Make PasswordAuthenticator number of hashing rounds configurable 
(CASSANDRA-8085)
 + * Fix AssertionError when binding nested collections in DELETE 
(CASSANDRA-8900)
 + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739)
 + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914)
 + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225)
 + * IndexSummary effectiveIndexInterval is now a guideline, not a rule 
(CASSANDRA-8993)
 + * Use correct bounds for page cache eviction of compressed files 
(CASSANDRA-8746)
 + * SSTableScanner enforces its bounds (CASSANDRA-8946)
 + * Cleanup cell equality (CASSANDRA-8947)
 + * Introduce intra-cluster message coalescing (CASSANDRA-8692)
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make

[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-04-01 Thread jake 
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a7b5df6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a7b5df6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a7b5df6b

Branch: refs/heads/trunk
Commit: a7b5df6b4c0942d41bd8df6e1b9c372535f210f4
Parents: b8717dc 28af4fa
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 11:14:47 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 11:14:47 2015 -0400

--

--

[1/2] cassandra git commit: fix import spacing

2015-04-01 Thread jake 
Repository: cassandra
Updated Branches:
  refs/heads/trunk b8717dc20 - a7b5df6b4


fix import spacing


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/28af4fa0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/28af4fa0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/28af4fa0

Branch: refs/heads/trunk
Commit: 28af4fa04fc84e61bf35bb3761c01f51533b9b15
Parents: 150f827
Author: T Jake Luciani j...@apache.org
Authored: Wed Apr 1 11:13:18 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Wed Apr 1 11:13:18 2015 -0400

--
 src/java/org/apache/cassandra/service/CassandraDaemon.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/28af4fa0/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java 
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index 412a0bf..b527528 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -26,7 +26,7 @@ import java.net.UnknownHostException;
 import java.rmi.registry.LocateRegistry;
 import java.rmi.server.RMIServerSocketFactory;
 import java.util.*;
-import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeUnit;
 import javax.management.MBeanServer;
 import javax.management.ObjectName;
 import javax.management.StandardMBean;

[jira] [Resolved] (CASSANDRA-5915) node flapping prevents replace_node from succeeding consistently

2015-04-01 Thread Brandon Williams (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-5915.
-
Resolution: Cannot Reproduce

replace_node doesn't even exist anymore, so closing.

 node flapping prevents replace_node from succeeding consistently
 

 Key: CASSANDRA-5915
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5915
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 1.2.8
Reporter: Chris Burroughs
 Attachments: cassandra.log.gz


 A node was down for a week or two due to hardware disk failure. I tried to 
 use replace_node to bring up a new node on the same physical host with the 
 same IPs. (rbranson suspected that using the same IP may be more issue 
 prone.) This failed due to unable to find sufficient sources for streaming 
 range  See CASSANDRA-5913 for a problem with how the failure was handled by 
 gossip.
 All of the other nodes should have been up the entire time, but when this 
 node came up it saw nodes flap up and down for quiet some time.  I was 
 eventually able to get replace_token to work by adding a 60 (!) second sleep 
 to StorageService:bootstrap.  I don't know if the right path is why are 
 things flapping so much or bootstrap should wait until things look stable.
 A few notes about the cluster:
  * 2 dc cluster (about 20 each), using GossipingPropertyFileSnitch
  * multi-dc no vpn setup: 
 http://mail-archives.apache.org/mod_mbox/cassandra-user/201306.mbox/%3c51bf5c79.7020...@gmail.com%3E
 Startup log from the successful (with sleep) replace_node attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

  1   2   3   >