[jira] [Commented] (CASSANDRA-10661) Integrate SASI to Cassandra

[Dave Brosius (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345812#comment-15345812
 ] 

Dave Brosius commented on CASSANDRA-10661:
--

AbstractTrie.equals isn't symmetric..

someAbstractTrie.equals(someMapEntry) could be true, but

someMapEntry.equals(someAbstractTrie) will always be false.

that is unstable.

> Integrate SASI to Cassandra
> ---
>
> Key: CASSANDRA-10661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10661
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Pavel Yaskevich
>Assignee: Pavel Yaskevich
>  Labels: sasi
> Fix For: 3.4
>
>
> We have recently released new secondary index engine 
> (https://github.com/xedin/sasi) build using SecondaryIndex API, there are 
> still couple of things to work out regarding 3.x since it's currently 
> targeted on 2.0 released. I want to make this an umbrella issue to all of the 
> things related to integration of SASI, which are also tracked in 
> [sasi_issues|https://github.com/xedin/sasi/issues], into mainline Cassandra 
> 3.x release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12076) Add username to AuthenticationException messages

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345795#comment-15345795
 ] 

Robert Stupp commented on CASSANDRA-12076:
--

Patch LGTM - [~beobal], ok with this?
Would go ahead and trigger CI for it then.

> Add username to AuthenticationException messages
> 
>
> Key: CASSANDRA-12076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12076
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Trivial
> Attachments: 12076-trunk.txt
>
>
> When an {{AuthenticationException}} is thrown, there are a few places where 
> the user that initiated the request is not included in the exception message. 
> It can be useful to have this information included for logging purposes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: use parameterized logging

[dbrosius]

Repository: cassandra
Updated Branches:
  refs/heads/trunk bf3255fc9 -> f147ca91c


use parameterized logging


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f147ca91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f147ca91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f147ca91

Branch: refs/heads/trunk
Commit: f147ca91c759b8013c5226774bc974c342bee3f0
Parents: bf3255f
Author: Dave Brosius 
Authored: Thu Jun 23 01:35:43 2016 -0400
Committer: Dave Brosius 
Committed: Thu Jun 23 01:36:23 2016 -0400

--
 src/java/org/apache/cassandra/config/Config.java | 2 +-
 .../org/apache/cassandra/db/WindowsFailedSnapshotTracker.java| 3 +--
 src/java/org/apache/cassandra/dht/RangeStreamer.java | 4 ++--
 .../org/apache/cassandra/dht/tokenallocator/TokenAllocation.java | 4 ++--
 src/java/org/apache/cassandra/io/util/FileUtils.java | 2 +-
 src/java/org/apache/cassandra/service/StartupChecks.java | 2 +-
 src/java/org/apache/cassandra/utils/WindowsTimer.java| 4 ++--
 7 files changed, 10 insertions(+), 11 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f147ca91/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index c18d1e0..2473c0c 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -459,6 +459,6 @@ public class Config
 configMap.put(name, value);
 }
 
-logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+logger.info("Node configuration:[{}]", Joiner.on("; 
").join(configMap.entrySet()));
 }
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f147ca91/src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java 
b/src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
index 7cc7893..134fb11 100644
--- a/src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
+++ b/src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
@@ -85,8 +85,7 @@ public class WindowsFailedSnapshotTracker
 }
 catch (IOException e)
 {
-logger.warn("Failed to open {}. Obsolete snapshots from 
previous runs will not be deleted.", TODELETEFILE);
-logger.warn("Exception: " + e);
+logger.warn("Failed to open {}. Obsolete snapshots from 
previous runs will not be deleted.", TODELETEFILE, e);
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f147ca91/src/java/org/apache/cassandra/dht/RangeStreamer.java
--
diff --git a/src/java/org/apache/cassandra/dht/RangeStreamer.java 
b/src/java/org/apache/cassandra/dht/RangeStreamer.java
index 8265e2b..ee2d792 100644
--- a/src/java/org/apache/cassandra/dht/RangeStreamer.java
+++ b/src/java/org/apache/cassandra/dht/RangeStreamer.java
@@ -326,8 +326,8 @@ public class RangeStreamer
 throw new IllegalStateException("Unable to find 
sufficient sources for streaming range " + range + " in keyspace " + keyspace + 
" with RF=1." +
 "If you want to ignore 
this, consider using system property 
-Dcassandra.consistent.rangemovement=false.");
 else
-logger.warn("Unable to find sufficient sources for 
streaming range " + range + " in keyspace " + keyspace + " with RF=1. " +
-"Keyspace might be missing data.");
+logger.warn("Unable to find sufficient sources for 
streaming range {} in keyspace {} with RF=1. " +
+"Keyspace might be missing data.", range, 
keyspace);
 }
 else
 throw new IllegalStateException("Unable to find sufficient 
sources for streaming range " + range + " in keyspace " + keyspace);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f147ca91/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java
--
diff --git 
a/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java 
b/src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java
index e715ff6..7658ea0 100644
--- 

cassandra git commit: Ninja fix broken commit from CASSANDRA-8142

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 82ef1cfc3 -> bf3255fc9


Ninja fix broken commit from CASSANDRA-8142


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bf3255fc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bf3255fc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bf3255fc

Branch: refs/heads/trunk
Commit: bf3255fc93db65b816b016958967003df38a6004
Parents: 82ef1cf
Author: Robert Stupp 
Authored: Thu Jun 23 07:34:25 2016 +0200
Committer: Robert Stupp 
Committed: Thu Jun 23 07:34:25 2016 +0200

--
 bin/cassandra | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf3255fc/bin/cassandra
--
diff --git a/bin/cassandra b/bin/cassandra
index d8daa5e..3206fdc 100755
--- a/bin/cassandra
+++ b/bin/cassandra
@@ -266,7 +266,7 @@ while true; do
 done
 
 if [ "x$allow_root" != "xyes" ] ; then
-if [ "`id -u`" = "1" ] || [ "`id -g`" = "0" ] ; then
+if [ "`id -u`" = "0" ] || [ "`id -g`" = "0" ] ; then
 echo "Running Cassandra as root user or group is not recommended - 
please start Cassandra using a different system user."
 echo "If you really want to force running Cassandra as root, use -R 
command line option."
 exit 1

[jira] [Issue Comment Deleted] (CASSANDRA-9779) Append-only optimization

[Jeff Jirsa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Jirsa updated CASSANDRA-9779:
--
Comment: was deleted

(was: {quote}
 I'm wondering if given DTCS and other optimization we have internally this 
really bring that much to the table.
{quote}


I've been watching this ticket primarily because if 'append only' tables exist, 
it should be possible to special case 
{{CompactionController.getFullyExpiredSSTables}} so that DTCS/TWCS can drop 
tables with timestamp overlaps (sstable expired blockers), which would be 
significantly more efficient than trying to split out read-repaired cells 
during compaction a la CASSANDRA-10496

)

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345726#comment-15345726
 ] 

Jeff Jirsa commented on CASSANDRA-9779:
---

{quote}
 I'm wondering if given DTCS and other optimization we have internally this 
really bring that much to the table.
{quote}


I've been watching this ticket primarily because if 'append only' tables exist, 
it should be possible to special case 
{{CompactionController.getFullyExpiredSSTables}} so that DTCS/TWCS can drop 
tables with timestamp overlaps (sstable expired blockers), which would be 
significantly more efficient than trying to split out read-repaired cells 
during compaction a la CASSANDRA-10496



> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12074) Faster check for open JMX port on startup

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345714#comment-15345714
 ] 

Jeff Jirsa commented on CASSANDRA-12074:


Concern here is that in real servers, there are other things that may show up 
in {{lsof -i}} other than running cassandra daemon - outbound jmx/nodetool 
connections (operator running commands against other servers, perhaps), or 
software bound to other interfaces will stop cassandra from starting.


> Faster check for open JMX port on startup
> -
>
> Key: CASSANDRA-12074
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12074
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 3.x
>
>
> Since CASSANDRA-7254, we check whether the JMX port is available before 
> starting Cassandra in order to provide a better error message when another 
> Cassandra process is already running.  The current check starts a Java 
> process to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, 
> which is basically instantaneous.
> By my estimate, this will shave about 40 minutes off our dtest runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11991) On clock skew, paxos may "corrupt" the node clock

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345678#comment-15345678
 ] 

sankalp kohli commented on CASSANDRA-11991:
---

+1 [~slebresne] Please commit it. 

> On clock skew, paxos may "corrupt" the node clock
> -
>
> Key: CASSANDRA-11991
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11991
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
>
> W made a mistake in CASSANDRA-9649 so that a temporal clock skew on one node 
> can "corrupt" other node clocks through Paxos. That wasn't intended and we 
> should fix that. I'll attach a patch later.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345631#comment-15345631
 ] 

Ariel Weisberg commented on CASSANDRA-12071:


I am seeing this against a single table. I don't think the number of tables 
matters. The memory pool is global for the process and shared by all tables.

The parallelism is reduced both for a single table and across tables. The issue 
is that there is a single threaded executor kicking off flushes that is waiting 
on the result of whatever amount of parallelism is available for a single 
Memtable flush. [The parallelism for a single Memtable flush is set to the the 
# of JBOD 
disks.|https://github.com/apache/cassandra/commit/e2c6341898fa43b0e262ef031f267587050b8d0f#diff-f0a15c3588b56c5ce53ece7c48e325b5R262]

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12043) Syncing most recent commit in CAS across replicas can cause all CAS queries in the CQL partition to fail

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345619#comment-15345619
 ] 

sankalp kohli commented on CASSANDRA-12043:
---

LGTM. I think we should get this reviewed through [~jasobrown] as well. 

> Syncing most recent commit in CAS across replicas can cause all CAS queries 
> in the CQL partition to fail
> 
>
> Key: CASSANDRA-12043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12043
> Project: Cassandra
>  Issue Type: Bug
>Reporter: sankalp kohli
>
> We update the most recent commit on requiredParticipant replicas if out of 
> sync during the prepare round in beginAndRepairPaxos method. We keep doing 
> this in a loop till the requiredParticipant replicas have the same most 
> recent commit or we hit timeout. 
> Say we have 3 machines A,B and C and gc grace on the table is 10 days. We do 
> a CAS write at time 0 and it went to A and B but not to C.  C will get the 
> hint later but will not update the most recent commit in paxos table. This is 
> how CAS hints work. 
> In the paxos table whose gc_grace=0, most_recent_commit in A and B will be 
> inserted with timestamp 0 and with a TTL of 10 days. After 10 days, this 
> insert will become a tombstone at time 0 till it is compacted away since 
> gc_grace=0.
> Do a CAS read after say 1 day on the same CQL partition and this time prepare 
> phase involved A and C. most_recent_commit on C for this CQL partition is 
> empty. A sends the most_recent_commit to C with a timestamp of 0 and with a 
> TTL of 10 days. This most_recent_commit on C will expire on 11th day since it 
> is inserted after 1 day. 
> most_recent_commit are now in sync on A,B and C, however A and B 
> most_recent_commit will expire on 10th day whereas for C it will expire on 
> 11th day since it was inserted one day later. 
> Do another CAS read after 10days when most_recent_commit on A and B have 
> expired and is treated as tombstones till compacted. In this CAS read, say A 
> and C are involved in prepare phase. most_recent_commit will not match 
> between them since it is expired in A and is still there on C. This will 
> cause most_recent_commit to be applied to A with a timestamp of 0 and TTL of 
> 10 days. If A has not compacted away the original most_recent_commit which 
> has expired, this new write to most_recent_commit wont be visible on reads 
> since there is a tombstone with same timestamp(Delete wins over data with 
> same timestamp). 
> Another round of prepare will follow and again A would say it does not know 
> about most_recent_write(covered by original write which is not a tombstone) 
> and C will again try to send the write to A. This can keep going on till the 
> request timeouts or only A and B are involved in the prepare phase. 
> When A’s original most_recent_commit which is now a tombstone is compacted, 
> all the inserts which it was covering will come live. This will in turn again 
> get played to another replica. This ping pong can keep going on for a long 
> time. 
> The issue is that most_recent_commit is expiring at different times across 
> replicas. When they get replayed to a replica to make it in sync, we again 
> set the TTL from that point.  
> During the CAS read which timed out, most_recent_commit was being sent to 
> another replica in a loop. Even in successful requests, it will try to loop 
> for a couple of times if involving A and C and then when the replicas which 
> respond are A and B, it will succeed. So this will have impact on latencies 
> as well. 
> These timeouts gets worse when a machine is down as no progress can be made 
> as the machine with unexpired commit is always involved in the CAS prepare 
> round. Also with range movements, the new machine gaining range has empty 
> most recent commit and gets the commit at a later time causing same issue. 
> Repro steps:
> 1. Paxos TTL is max(3 hours, gc_grace) as defined in 
> SystemKeyspace.paxosTtl(). Change this method to not put a minimum TTL of 3 
> hours. 
> Method  SystemKeyspace.paxosTtl() will look like return 
> metadata.getGcGraceSeconds();   instead of return Math.max(3 * 3600, 
> metadata.getGcGraceSeconds());
> We are doing this so that we dont need to wait for 3 hours. 
> Create a 3 node cluster with the code change suggested above with machines 
> A,B and C
> CREATE KEYSPACE  test WITH REPLICATION = { 'class' : 'SimpleStrategy', 
> 'replication_factor' : 3 };
> use test;
> CREATE TABLE users (a int PRIMARY KEY,b int);
> alter table users WITH gc_grace_seconds=120;
> consistency QUORUM;
> bring down machine C
> INSERT INTO users (user_name, password ) VALUES ( 1,1) IF NOT EXISTS;
> Nodetool flush on machine A and B
> Bring up the down machine B 
> consistency SERIAL;
> 

[jira] [Commented] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345605#comment-15345605
 ] 

Jonathan Ellis commented on CASSANDRA-12071:


You're seeing this against a single table, right?  So ingest parallelism across 
multiple tables would make it even worse?

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12035) Structure for tpstats output (JSON, YAML)

[Hiroyuki Nishi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hiroyuki Nishi updated CASSANDRA-12035:
---
Status: Patch Available  (was: Open)

> Structure for tpstats output (JSON, YAML)
> -
>
> Key: CASSANDRA-12035
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12035
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Hiroyuki Nishi
>Assignee: Hiroyuki Nishi
>Priority: Minor
> Attachments: CASSANDRA-12035-trunk.patch, 
> tablestats_sample_result.json, tablestats_sample_result.txt, 
> tablestats_sample_result.yaml, tpstats_sample_result.json, 
> tpstats_sample_result.txt, tpstats_sample_result.yaml
>
>
> In CASSANDRA-5977, some extra output formats such as JSON and YAML were added 
> for nodetool tablestats. 
> Similarly, I would like to add the output formats in nodetool tpstats.
> Also, I tried to refactor the tablestats's code about the output formats to 
> integrate the existing code with my code.
> Please review the attached patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11976) cqlsh tab completion doesn't work in 2.1

[Yusuke Takata (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yusuke Takata updated CASSANDRA-11976:
--
Status: Patch Available  (was: Open)

> cqlsh tab completion doesn't work in 2.1
> 
>
> Key: CASSANDRA-11976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11976
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Yusuke Takata
>Priority: Minor
>  Labels: cqlsh
> Attachments: CASSANDRA-11976-1.patch, CASSANDRA-11976-2.patch
>
>
> cqlsh tab completion doesn't work when there are two tables with the same 
> prefix.
> I found the similar completion issue at CASSANDRA-10733, but my problem is 
> not fixed by the existing issue in 2.1.
> {code}
> cqlsh> CREATE KEYSPACE sample_ks WITH replication = {'class': 
> 'SimpleStrategy' , 'replication_factor': 1 };
> cqlsh> CREATE TABLE sample_ks.tbl_a ( key text PRIMARY KEY );
> cqlsh> CREATE TABLE sample_ks.tbl_b ( key text PRIMARY KEY, value int );
> // works correctly
> cqlsh> INSERT INTO sample_ks.tb
> cqlsh> INSERT INTO sample_ks.tbl_
> // fix required
> cqlsh> INSERT INTO samp
> cqlsh> INSERT INTO sample_ks.tbl_( 
> {code}
> Also, completion doesn't work with a single column table.
> {code}
> cqlsh> CREATE KEYSPACE sample_ks WITH replication = {'class': 
> 'SimpleStrategy' , 'replication_factor': 1 };
> cqlsh> CREATE TABLE sample_ks.tbl_a ( key text PRIMARY KEY );
> cqlsh> CREATE TABLE sample_ks.tbl_b ( key text PRIMARY KEY, value int );
> // fix required (unnecessary comma)
> cqlsh> INSERT INTO sample_ks.tbl_a
> cqlsh> INSERT INTO sample_ks.tbl_a (key,
> // fix required (no reaction)
> cqlsh> INSERT INTO sample_ks.tbl_a (key) VALU
> cqlsh> INSERT INTO sample_ks.tbl_a (key) VALU
> // fix required (I can't insert only a key.)
> cqlsh> INSERT INTO sample_ks.tbl_b
> cqlsh> INSERT INTO sample_ks.tbl_b (key, value
> {code}
> I fixed the completion problem in 2.1 branch. Could someone review the 
> attached patch?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12076) Add username to AuthenticationException messages

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-12076:

Status: Patch Available  (was: Open)

> Add username to AuthenticationException messages
> 
>
> Key: CASSANDRA-12076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12076
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Trivial
> Attachments: 12076-trunk.txt
>
>
> When an {{AuthenticationException}} is thrown, there are a few places where 
> the user that initiated the request is not included in the exception message. 
> It can be useful to have this information included for logging purposes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12076) Add username to AuthenticationException messages

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-12076:

Attachment: 12076-trunk.txt

> Add username to AuthenticationException messages
> 
>
> Key: CASSANDRA-12076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12076
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Trivial
> Attachments: 12076-trunk.txt
>
>
> When an {{AuthenticationException}} is thrown, there are a few places where 
> the user that initiated the request is not included in the exception message. 
> It can be useful to have this information included for logging purposes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12076) Add username to AuthenticationException messages

[Geoffrey Yu (JIRA)]

Geoffrey Yu created CASSANDRA-12076:
---

 Summary: Add username to AuthenticationException messages
 Key: CASSANDRA-12076
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12076
 Project: Cassandra
  Issue Type: Improvement
Reporter: Geoffrey Yu
Assignee: Geoffrey Yu
Priority: Trivial


When an {{AuthenticationException}} is thrown, there are a few places where the 
user that initiated the request is not included in the exception message. It 
can be useful to have this information included for logging purposes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12075) Include whether or not the client should retry the request when throwing a RequestExecutionException

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-12075:

Issue Type: Improvement  (was: New Feature)

> Include whether or not the client should retry the request when throwing a 
> RequestExecutionException
> 
>
> Key: CASSANDRA-12075
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12075
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
>
> Some requests that result in an error should not be retried by the client. 
> Right now if the client gets an error, it has no way of knowing whether or 
> not it should retry. We can include an extra field in each 
> {{RequestExecutionException}} that will indicate whether the client should 
> retry, retry on a different host, or not retry at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12075) Include whether or not the client should retry the request when throwing a RequestExecutionException

[Geoffrey Yu (JIRA)]

Geoffrey Yu created CASSANDRA-12075:
---

 Summary: Include whether or not the client should retry the 
request when throwing a RequestExecutionException
 Key: CASSANDRA-12075
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12075
 Project: Cassandra
  Issue Type: New Feature
Reporter: Geoffrey Yu
Assignee: Geoffrey Yu
Priority: Minor


Some requests that result in an error should not be retried by the client. 
Right now if the client gets an error, it has no way of knowing whether or not 
it should retry. We can include an extra field in each 
{{RequestExecutionException}} that will indicate whether the client should 
retry, retry on a different host, or not retry at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11880) Display number of tables in cfstats

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-11880:

Attachment: 11880-trunk.txt

> Display number of tables in cfstats
> ---
>
> Key: CASSANDRA-11880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11880
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
> Attachments: 11880-trunk.txt
>
>
> We should display the number of tables in a Cassandra cluster in {{nodetool 
> cfstats}}. This would be useful for monitoring.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11880) Display number of tables in cfstats

[Geoffrey Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Yu updated CASSANDRA-11880:

Status: Patch Available  (was: Open)

> Display number of tables in cfstats
> ---
>
> Key: CASSANDRA-11880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11880
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
> Attachments: 11880-trunk.txt
>
>
> We should display the number of tables in a Cassandra cluster in {{nodetool 
> cfstats}}. This would be useful for monitoring.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12074) Faster check for open JMX port on startup

[Jeff Jirsa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345329#comment-15345329
 ] 

Jeff Jirsa commented on CASSANDRA-12074:


Seems to work on OSX:

{code}
bash-3.2# ccm status
Cluster: 'test'
---
node1: UP
bash-3.2# netstat -lna | grep 7100
tcp4   0  0  127.0.0.1.7100 127.0.0.1.57741ESTABLISHED
tcp4   0  0  127.0.0.1.57741127.0.0.1.7100 ESTABLISHED
tcp4   0  0  127.0.0.1.7100 *.*LISTEN
bash-3.2# lsof -i ":7100" ; if [ $? -eq "0" ]; then echo "in use?"; else echo 
"not in use"; fi
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java5895 root   92u  IPv4 0xd9ed09d8f5e70fc1  0t0  TCP 
localhost:font-service (LISTEN)
in use?
bash-3.2#
{code}



> Faster check for open JMX port on startup
> -
>
> Key: CASSANDRA-12074
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12074
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 3.x
>
>
> Since CASSANDRA-7254, we check whether the JMX port is available before 
> starting Cassandra in order to provide a better error message when another 
> Cassandra process is already running.  The current check starts a Java 
> process to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, 
> which is basically instantaneous.
> By my estimate, this will shave about 40 minutes off our dtest runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12074) Faster check for open JMX port on startup

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345235#comment-15345235
 ] 

Tyler Hobbs commented on CASSANDRA-12074:
-

Trunk patch and dtest run:

||branch||dtest||
|[CASSANDRA-12074|https://github.com/thobbs/cassandra/tree/CASSANDRA-12074]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12074-dtest]|

Can somebody on OSX review this to ensure the {{lsof}} command works properly 
there?

> Faster check for open JMX port on startup
> -
>
> Key: CASSANDRA-12074
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12074
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 3.x
>
>
> Since CASSANDRA-7254, we check whether the JMX port is available before 
> starting Cassandra in order to provide a better error message when another 
> Cassandra process is already running.  The current check starts a Java 
> process to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, 
> which is basically instantaneous.
> By my estimate, this will shave about 40 minutes off our dtest runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12074) Faster check for open JMX port on startup

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12074:

Status: Patch Available  (was: Open)

> Faster check for open JMX port on startup
> -
>
> Key: CASSANDRA-12074
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12074
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 3.x
>
>
> Since CASSANDRA-7254, we check whether the JMX port is available before 
> starting Cassandra in order to provide a better error message when another 
> Cassandra process is already running.  The current check starts a Java 
> process to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, 
> which is basically instantaneous.
> By my estimate, this will shave about 40 minutes off our dtest runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12074) Faster check for open JMX port on startup

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12074:

Description: 
Since CASSANDRA-7254, we check whether the JMX port is available before 
starting Cassandra in order to provide a better error message when another 
Cassandra process is already running.  The current check starts a Java process 
to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, which is 
basically instantaneous.

By my estimate, this will shave about 40 minutes off our dtest runs.

  was:
Since CASSANDRA-7245, we check whether the JMX port is available before 
starting Cassandra in order to provide a better error message when another 
Cassandra process is already running.  The current check starts a Java process 
to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, which is 
basically instantaneous.

By my estimate, this will shave about 40 minutes off our dtest runs.


> Faster check for open JMX port on startup
> -
>
> Key: CASSANDRA-12074
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12074
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Lifecycle
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 3.x
>
>
> Since CASSANDRA-7254, we check whether the JMX port is available before 
> starting Cassandra in order to provide a better error message when another 
> Cassandra process is already running.  The current check starts a Java 
> process to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, 
> which is basically instantaneous.
> By my estimate, this will shave about 40 minutes off our dtest runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12074) Faster check for open JMX port on startup

[Tyler Hobbs (JIRA)]

Tyler Hobbs created CASSANDRA-12074:
---

 Summary: Faster check for open JMX port on startup
 Key: CASSANDRA-12074
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12074
 Project: Cassandra
  Issue Type: Improvement
  Components: Lifecycle
Reporter: Tyler Hobbs
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 3.x


Since CASSANDRA-7245, we check whether the JMX port is available before 
starting Cassandra in order to provide a better error message when another 
Cassandra process is already running.  The current check starts a Java process 
to do this, which takes ~3 seconds.  Instead, we can use {{lsof}}, which is 
basically instantaneous.

By my estimate, this will shave about 40 minutes off our dtest runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12073) [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial results

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-12073:

Reproduced In: 3.7

> [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial 
> results
> ---
>
> Key: CASSANDRA-12073
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12073
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.7
>Reporter: DOAN DuyHai
>
> {noformat}
> cqlsh:music> CREATE TABLE music.albums (
> id uuid PRIMARY KEY,
> artist text,
> country text,
> quality text,
> status text,
> title text,
> year int
> );
> cqlsh:music> CREATE CUSTOM INDEX albums_artist_idx ON music.albums (artist) 
> USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 
> 'CONTAINS', 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> cqlsh:music> SELECT * FROM albums WHERE artist like 'lady%'  LIMIT 100;
>  id   | artist| country| quality 
> | status| title | year
> --+---++-+---+---+--
>  372bb0ab-3263-41bc-baad-bb520ddfa787 | Lady Gaga |USA |  normal 
> |  Official |   Red and Blue EP | 2006
>  1a4abbcd-b5de-4c69-a578-31231e01ff09 | Lady Gaga |Unknown |  normal 
> | Promotion |Poker Face | 2008
>  31f4a0dc-9efc-48bf-9f5e-bfc09af42b82 | Lady Gaga |USA |  normal 
> |  Official |   The Cherrytree Sessions | 2009
>  8ebfaebd-28d0-477d-b735-469661ce6873 | Lady Gaga |Unknown |  normal 
> |  Official |Poker Face | 2009
>  98107d82-e0dd-46bc-a273-1577578984c7 | Lady Gaga |USA |  normal 
> |  Official |   Just Dance: The Remixes | 2008
>  a76af0f2-f5c5-4306-974a-e3c17158e6c6 | Lady Gaga |  Italy |  normal 
> |  Official |  The Fame | 2008
>  849ee019-8b15-4767-8660-537ab9710459 | Lady Gaga |USA |  normal 
> |  Official |Christmas Tree | 2008
>  4bad59ac-913f-43da-9d48-89adc65453d2 | Lady Gaga |  Australia |  normal 
> |  Official | Eh Eh | 2009
>  80327731-c450-457f-bc12-0a8c21fd9c5d | Lady Gaga |USA |  normal 
> |  Official | Just Dance Remixes Part 2 | 2008
>  3ad33659-e932-4d31-a040-acab0e23c3d4 | Lady Gaga |Unknown |  normal 
> |  null |Just Dance | 2008
>  9adce7f6-6a1d-49fd-b8bd-8f6fac73558b | Lady Gaga | United Kingdom |  normal 
> |  Official |Just Dance | 2009
> (11 rows)
> {noformat}
> *SASI* says that there are only 11 artists whose name starts with {{lady}}.
> However, in the data set, there are:
> * Lady Pank
> * Lady Saw
> * Lady Saw
> * Ladyhawke
> * Ladytron
> * Ladysmith Black Mambazo
> * Lady Gaga
> * Lady Sovereign
> etc ...
> By debugging the source code, the issue is in 
> {{OnDiskIndex.TermIterator::computeNext()}}
> {code:java}
> for (;;)
> {
> if (currentBlock == null)
> return endOfData();
> if (offset >= 0 && offset < currentBlock.termCount())
> {
> DataTerm currentTerm = currentBlock.getTerm(nextOffset());
> if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
> continue;
> // flip the flag right on the first bounds match
> // to avoid expensive comparisons
> checkLower = false;
> if (checkUpper && !e.isUpperSatisfiedBy(currentTerm))
> return endOfData();
> return currentTerm;
> }
> nextBlock();
> }
> {code}
>  So the {{endOfData()}} conditions are:
> * currentBlock == null
> * checkUpper && !e.isUpperSatisfiedBy(currentTerm)
> The problem is that {{e::isUpperSatisfiedBy}} is checking not only whether 
> the term match but also returns *false* when it's a *partial term* !
> {code:java}
> public boolean isUpperSatisfiedBy(OnDiskIndex.DataTerm term)
> {
> if (!hasUpper())
> return true;
> if (nonMatchingPartial(term))
> return false;
> int cmp = term.compareTo(validator, upper.value, false);
> return cmp < 0 || cmp == 0 && upper.inclusive;
> }
> {code}
> By debugging the OnDiskIndex data, I've found:
> {noformat}
> ...
> Data Term (partial ? false) : lady gaga. 0x0, TokenTree offset : 21120
> Data Term (partial ? true) : lady of bells. 0x0, TokenTree offset : 21360
> Data Term (partial ? false) : lady pank. 

[jira] [Created] (CASSANDRA-12073) [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial results

[DOAN DuyHai (JIRA)]

DOAN DuyHai created CASSANDRA-12073:
---

 Summary: [SASI] PREFIX search on CONTAINS/NonTokenizer mode 
returns only partial results
 Key: CASSANDRA-12073
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12073
 Project: Cassandra
  Issue Type: Bug
  Components: CQL
 Environment: Cassandra 3.7
Reporter: DOAN DuyHai


{noformat}
cqlsh:music> CREATE TABLE music.albums (
id uuid PRIMARY KEY,
artist text,
country text,
quality text,
status text,
title text,
year int
);

cqlsh:music> CREATE CUSTOM INDEX albums_artist_idx ON music.albums (artist) 
USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 
'CONTAINS', 'analyzer_class': 
'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
'case_sensitive': 'false'};

cqlsh:music> SELECT * FROM albums WHERE artist like 'lady%'  LIMIT 100;

 id   | artist| country| quality | 
status| title | year
--+---++-+---+---+--
 372bb0ab-3263-41bc-baad-bb520ddfa787 | Lady Gaga |USA |  normal |  
Official |   Red and Blue EP | 2006
 1a4abbcd-b5de-4c69-a578-31231e01ff09 | Lady Gaga |Unknown |  normal | 
Promotion |Poker Face | 2008
 31f4a0dc-9efc-48bf-9f5e-bfc09af42b82 | Lady Gaga |USA |  normal |  
Official |   The Cherrytree Sessions | 2009
 8ebfaebd-28d0-477d-b735-469661ce6873 | Lady Gaga |Unknown |  normal |  
Official |Poker Face | 2009
 98107d82-e0dd-46bc-a273-1577578984c7 | Lady Gaga |USA |  normal |  
Official |   Just Dance: The Remixes | 2008
 a76af0f2-f5c5-4306-974a-e3c17158e6c6 | Lady Gaga |  Italy |  normal |  
Official |  The Fame | 2008
 849ee019-8b15-4767-8660-537ab9710459 | Lady Gaga |USA |  normal |  
Official |Christmas Tree | 2008
 4bad59ac-913f-43da-9d48-89adc65453d2 | Lady Gaga |  Australia |  normal |  
Official | Eh Eh | 2009
 80327731-c450-457f-bc12-0a8c21fd9c5d | Lady Gaga |USA |  normal |  
Official | Just Dance Remixes Part 2 | 2008
 3ad33659-e932-4d31-a040-acab0e23c3d4 | Lady Gaga |Unknown |  normal |  
null |Just Dance | 2008
 9adce7f6-6a1d-49fd-b8bd-8f6fac73558b | Lady Gaga | United Kingdom |  normal |  
Official |Just Dance | 2009

(11 rows)
{noformat}

*SASI* says that there are only 11 artists whose name starts with {{lady}}.

However, in the data set, there are:

* Lady Pank
* Lady Saw
* Lady Saw
* Ladyhawke
* Ladytron
* Ladysmith Black Mambazo
* Lady Gaga
* Lady Sovereign

etc ...

By debugging the source code, the issue is in 
{{OnDiskIndex.TermIterator::computeNext()}}

{code:java}
for (;;)
{
if (currentBlock == null)
return endOfData();

if (offset >= 0 && offset < currentBlock.termCount())
{
DataTerm currentTerm = currentBlock.getTerm(nextOffset());
if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
continue;

// flip the flag right on the first bounds match
// to avoid expensive comparisons
checkLower = false;

if (checkUpper && !e.isUpperSatisfiedBy(currentTerm))
return endOfData();
return currentTerm;
}
nextBlock();
}
{code}

 So the {{endOfData()}} conditions are:

* currentBlock == null
* checkUpper && !e.isUpperSatisfiedBy(currentTerm)

The problem is that {{e::isUpperSatisfiedBy}} is checking not only whether the 
term match but also returns *false* when it's a *partial term* !

{code:java}
public boolean isUpperSatisfiedBy(OnDiskIndex.DataTerm term)
{
if (!hasUpper())
return true;

if (nonMatchingPartial(term))
return false;

int cmp = term.compareTo(validator, upper.value, false);
return cmp < 0 || cmp == 0 && upper.inclusive;
}
{code}

By debugging the OnDiskIndex data, I've found:

{noformat}
...
Data Term (partial ? false) : lady gaga. 0x0, TokenTree offset : 21120
Data Term (partial ? true) : lady of bells. 0x0, TokenTree offset : 21360
Data Term (partial ? false) : lady pank. 0x0, TokenTree offset : 21440
...
{noformat}

*lady gaga* did match but then *lady of bells* fails because it is a partial 
match. Then SASI returns {{endOfData()}} instead continuing to check *lady pank*

One quick & dirty fix I've found is to add 
{{!e.nonMatchingPartial(currentTerm)}} to the *if* condition

{code:java}
 if (checkUpper && !e.isUpperSatisfiedBy(currentTerm) && 

[jira] [Created] (CASSANDRA-12072) dtest failure in auth_test.TestAuthRoles.udf_permissions_in_selection_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12072:
-

 Summary: dtest failure in 
auth_test.TestAuthRoles.udf_permissions_in_selection_test
 Key: CASSANDRA-12072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12072
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

Multiple failures:

http://cassci.datastax.com/job/trunk_offheap_dtest/265/testReport/auth_test/TestAuthRoles/udf_permissions_in_selection_test

http://cassci.datastax.com/job/trunk_offheap_dtest/265/testReport/auth_test/TestAuthRoles/create_and_grant_roles_with_superuser_status_test/

http://cassci.datastax.com/job/trunk_offheap_dtest/265/testReport/auth_test/TestAuthRoles/drop_keyspace_cleans_up_function_level_permissions_test/

http://cassci.datastax.com/job/trunk_offheap_dtest/265/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_max_parse_errors/

http://cassci.datastax.com/job/trunk_offheap_dtest/265/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_read_wrong_column_names/

http://cassci.datastax.com/job/trunk_offheap_dtest/264/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_boolstyle_round_trip/

http://cassci.datastax.com/job/trunk_offheap_dtest/264/testReport/compaction_test/TestCompaction_with_SizeTieredCompactionStrategy/disable_autocompaction_alter_test/

http://cassci.datastax.com/job/trunk_offheap_dtest/264/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_describe/

http://cassci.datastax.com/job/trunk_offheap_dtest/264/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_describe_mv/

Logs are from 
http://cassci.datastax.com/job/trunk_offheap_dtest/265/testReport/auth_test/TestAuthRoles/udf_permissions_in_selection_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12056) dtest failure in pushed_notifications_test.TestPushedNotifications.add_and_remove_node_test

[Sean McCarthy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean McCarthy resolved CASSANDRA-12056.
---
Resolution: Fixed
  Reviewer: Sean McCarthy

https://github.com/riptano/cassandra-dtest/pull/1053

> dtest failure in 
> pushed_notifications_test.TestPushedNotifications.add_and_remove_node_test
> ---
>
> Key: CASSANDRA-12056
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12056
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/482/testReport/pushed_notifications_test/TestPushedNotifications/add_and_remove_node_test
> Failed on CassCI build cassandra-2.1_dtest #482
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/pushed_notifications_test.py", line 
> 242, in add_and_remove_node_test
> self.assertEquals(2, len(notifications), notifications)
>   File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
>   File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> "[{'change_type': u'NEW_NODE', 'address': ('127.0.0.2', 9042)}]
> {code}
> Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12071:

Assignee: Marcus Eriksson

> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>Assignee: Marcus Eriksson
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Ariel Weisberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ariel Weisberg updated CASSANDRA-12071:
---
Description: 
The way flushing used to work is that a ColumnFamilyStore could have multiple 
Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
same time. The way it works now there can be only a single flush of any 
ColumnFamilyStore & Memtable running in the C* process, and the number of 
threads applied to that flush is bounded by the number of disks in JBOD.

This works ok most of the time but occasionally flushing will be a little 
slower and ingest will outstrip it and then block on available memory. At this 
point you see several second stalls that cause timeouts.

This is a problem for reasonable configurations that don't use JBOD but have 
access to a fast disk that can handle some IO queuing (RAID, SSD).

You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, SSD) 
if you unthrottle compaction or set it to something like 64 megabytes/second 
and run with 8 compaction threads and stress with the default write workload 
and a reasonable number of threads. I tested with 96.

It started happening after about 60 gigabytes of data was loaded.

  was:
The way flushing used to work is that a ColumnFamilyStore could have multiple 
memtables flushing at once. The way it works now there can be only a single 
flush of any memtable running in the C* process, and the number of threads 
applied to that flush is bounded by the number of disks in JBOD.

This works ok most of the time but occasionally flushing will be a little 
slower and ingest will outstrip it and then block on available memory. At this 
point you see several second stalls that cause timeouts.

This is a problem for reasonable configurations that don't use JBOD but have 
access to a fast disk that can handle some IO queuing (RAID, SSD).

You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, SSD) 
if you unthrottle compaction or set it to something like 64 megabytes/second 
and run with 8 compaction threads and stress with the default write workload 
and a reasonable number of threads. I tested with 96.

It started happening after about 60 gigabytes of data was loaded.


> Regression in flushing throughput under load after CASSANDRA-6696
> -
>
> Key: CASSANDRA-12071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Ariel Weisberg
>
> The way flushing used to work is that a ColumnFamilyStore could have multiple 
> Memtables flushing at once and multiple ColumnFamilyStores could flush at the 
> same time. The way it works now there can be only a single flush of any 
> ColumnFamilyStore & Memtable running in the C* process, and the number of 
> threads applied to that flush is bounded by the number of disks in JBOD.
> This works ok most of the time but occasionally flushing will be a little 
> slower and ingest will outstrip it and then block on available memory. At 
> this point you see several second stalls that cause timeouts.
> This is a problem for reasonable configurations that don't use JBOD but have 
> access to a fast disk that can handle some IO queuing (RAID, SSD).
> You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, 
> SSD) if you unthrottle compaction or set it to something like 64 
> megabytes/second and run with 8 compaction threads and stress with the 
> default write workload and a reasonable number of threads. I tested with 96.
> It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12071) Regression in flushing throughput under load after CASSANDRA-6696

[Ariel Weisberg (JIRA)]

Ariel Weisberg created CASSANDRA-12071:
--

 Summary: Regression in flushing throughput under load after 
CASSANDRA-6696
 Key: CASSANDRA-12071
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12071
 Project: Cassandra
  Issue Type: Bug
  Components: Local Write-Read Paths
Reporter: Ariel Weisberg


The way flushing used to work is that a ColumnFamilyStore could have multiple 
memtables flushing at once. The way it works now there can be only a single 
flush of any memtable running in the C* process, and the number of threads 
applied to that flush is bounded by the number of disks in JBOD.

This works ok most of the time but occasionally flushing will be a little 
slower and ingest will outstrip it and then block on available memory. At this 
point you see several second stalls that cause timeouts.

This is a problem for reasonable configurations that don't use JBOD but have 
access to a fast disk that can handle some IO queuing (RAID, SSD).

You can reproduce on beefy hardware (12 cores 24 threads, 64 gigs of RAM, SSD) 
if you unthrottle compaction or set it to something like 64 megabytes/second 
and run with 8 compaction threads and stress with the default write workload 
and a reasonable number of threads. I tested with 96.

It started happening after about 60 gigabytes of data was loaded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12064) dtest failure in auth_test.TestAuth.login_test

[Sean McCarthy (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345125#comment-15345125
 ] 

Sean McCarthy commented on CASSANDRA-12064:
---

[~beobal], we have this in-flight: 
https://github.com/riptano/cassandra-dtest/pull/1012

> dtest failure in auth_test.TestAuth.login_test
> --
>
> Key: CASSANDRA-12064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12064
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1284/testReport/auth_test/TestAuth/login_test
> Failed on CassCI build trunk_dtest #1284
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/auth_test.py", line 82, in login_test
> assert isinstance(e.errors.values()[0], AuthenticationFailed)
> {code}
> Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12070) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12070:

Reviewer: Stefania
  Status: Patch Available  (was: Open)

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template
> -
>
> Key: CASSANDRA-12070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12070
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/262/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_data_validation_on_read_template
> Failed on CassCI build trunk_offheap_dtest #262
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 1608, in test_data_validation_on_read_template
> self.assertFalse(err)
>   File "/usr/lib/python2.7/unittest/case.py", line 416, in assertFalse
> raise self.failureException(msg)
> '\'Process ImportProcess-3:\\nTraceback (most recent call last):\\n  File 
> "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap\\n   
>  self.run()\\n  File 
> "/home/automaton/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 2205, in 
> run\\nself.report_error(exc)\\nTypeError: report_error() takes at least 3 
> arguments (2 given)\\n\' is not false
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-12070) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs reassigned CASSANDRA-12070:
---

Assignee: Tyler Hobbs  (was: DS Test Eng)

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template
> -
>
> Key: CASSANDRA-12070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12070
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Tyler Hobbs
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/262/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_data_validation_on_read_template
> Failed on CassCI build trunk_offheap_dtest #262
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 1608, in test_data_validation_on_read_template
> self.assertFalse(err)
>   File "/usr/lib/python2.7/unittest/case.py", line 416, in assertFalse
> raise self.failureException(msg)
> '\'Process ImportProcess-3:\\nTraceback (most recent call last):\\n  File 
> "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap\\n   
>  self.run()\\n  File 
> "/home/automaton/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 2205, in 
> run\\nself.report_error(exc)\\nTypeError: report_error() takes at least 3 
> arguments (2 given)\\n\' is not false
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12070) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345086#comment-15345086
 ] 

Tyler Hobbs commented on CASSANDRA-12070:
-

Patch and pending CI runs (merge to trunk was trivial):

||branch||dtest||
|[CASSANDRA-12070-3.0|https://github.com/thobbs/cassandra/tree/CASSANDRA-12070-3.0]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12070-3.0-dtest]|
|[CASSANDRA-12070-trunk|https://github.com/thobbs/cassandra/tree/CASSANDRA-12070-trunk]|[dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-12070-trunk-dtest]|

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template
> -
>
> Key: CASSANDRA-12070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12070
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/262/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_data_validation_on_read_template
> Failed on CassCI build trunk_offheap_dtest #262
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 1608, in test_data_validation_on_read_template
> self.assertFalse(err)
>   File "/usr/lib/python2.7/unittest/case.py", line 416, in assertFalse
> raise self.failureException(msg)
> '\'Process ImportProcess-3:\\nTraceback (most recent call last):\\n  File 
> "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap\\n   
>  self.run()\\n  File 
> "/home/automaton/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 2205, in 
> run\\nself.report_error(exc)\\nTypeError: report_error() takes at least 3 
> arguments (2 given)\\n\' is not false
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345077#comment-15345077
 ] 

Joshua McKenzie commented on CASSANDRA-9318:


[~aweisberg]: You've spent a fair amount of time thinking about this problem - 
would you be up for reviewing this?

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
> Attachments: backpressure.png, limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Sergio Bossa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergio Bossa updated CASSANDRA-9318:

Attachment: limit.btm
backpressure.png
no_backpressure.png

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
> Attachments: backpressure.png, limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12070) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-12070:

Since Version: 3.0.5
Fix Version/s: 3.x
   3.0.x

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template
> -
>
> Key: CASSANDRA-12070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12070
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/262/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_data_validation_on_read_template
> Failed on CassCI build trunk_offheap_dtest #262
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 1608, in test_data_validation_on_read_template
> self.assertFalse(err)
>   File "/usr/lib/python2.7/unittest/case.py", line 416, in assertFalse
> raise self.failureException(msg)
> '\'Process ImportProcess-3:\\nTraceback (most recent call last):\\n  File 
> "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap\\n   
>  self.run()\\n  File 
> "/home/automaton/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 2205, in 
> run\\nself.report_error(exc)\\nTypeError: report_error() takes at least 3 
> arguments (2 given)\\n\' is not false
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12070) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344993#comment-15344993
 ] 

Tyler Hobbs commented on CASSANDRA-12070:
-

This looks like a bug in the error handling of {{copyutil.py}}.  I can take 
this ticket.

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template
> -
>
> Key: CASSANDRA-12070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12070
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/262/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_data_validation_on_read_template
> Failed on CassCI build trunk_offheap_dtest #262
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 1608, in test_data_validation_on_read_template
> self.assertFalse(err)
>   File "/usr/lib/python2.7/unittest/case.py", line 416, in assertFalse
> raise self.failureException(msg)
> '\'Process ImportProcess-3:\\nTraceback (most recent call last):\\n  File 
> "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap\\n   
>  self.run()\\n  File 
> "/home/automaton/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 2205, in 
> run\\nself.report_error(exc)\\nTypeError: report_error() takes at least 3 
> arguments (2 given)\\n\' is not false
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-9318:
---
Status: Patch Available  (was: Reopened)

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Jeremiah Jordan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan reopened CASSANDRA-9318:

  Assignee: Sergio Bossa  (was: Ariel Weisberg)

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Sergio Bossa
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12063) Brotli storage compression

[Eric Evans (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Evans updated CASSANDRA-12063:
---
Fix Version/s: 2.2.x

> Brotli storage compression
> --
>
> Key: CASSANDRA-12063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12063
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Eric Evans
>Assignee: Eric Evans
>Priority: Minor
> Fix For: 2.2.x
>
>
> Brotli is a compression algorithm based on a modern variant of the LZ77 
> algorithm, Huffman coding and 2nd order context modeling.  It produces 
> smaller compressed sizes at costs comparable to deflate.
> I have a working [ICompressor 
> implementation|https://github.com/eevans/cassandra-brotli] which has received 
> a fair amount of testing already. I'll follow up shortly with a Cassandra 
> changeset(s) for review.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12070) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12070:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_data_validation_on_read_template
 Key: CASSANDRA-12070
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12070
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_offheap_dtest/262/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_data_validation_on_read_template

Failed on CassCI build trunk_offheap_dtest #262

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
1608, in test_data_validation_on_read_template
self.assertFalse(err)
  File "/usr/lib/python2.7/unittest/case.py", line 416, in assertFalse
raise self.failureException(msg)
'\'Process ImportProcess-3:\\nTraceback (most recent call last):\\n  File 
"/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap\\n
self.run()\\n  File 
"/home/automaton/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 2205, in 
run\\nself.report_error(exc)\\nTypeError: report_error() takes at least 3 
arguments (2 given)\\n\' is not false
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7937) Apply backpressure gently when overloaded with writes

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344961#comment-15344961
 ] 

Sergio Bossa commented on CASSANDRA-7937:
-

[~weideng], see: 
https://issues.apache.org/jira/browse/CASSANDRA-9318?focusedCommentId=15344958=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15344958

> Apply backpressure gently when overloaded with writes
> -
>
> Key: CASSANDRA-7937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7937
> Project: Cassandra
>  Issue Type: Improvement
> Environment: Cassandra 2.0
>Reporter: Piotr Kołaczkowski
>  Labels: performance
>
> When writing huge amounts of data into C* cluster from analytic tools like 
> Hadoop or Apache Spark, we can see that often C* can't keep up with the load. 
> This is because analytic tools typically write data "as fast as they can" in 
> parallel, from many nodes and they are not artificially rate-limited, so C* 
> is the bottleneck here. Also, increasing the number of nodes doesn't really 
> help, because in a collocated setup this also increases number of 
> Hadoop/Spark nodes (writers) and although possible write performance is 
> higher, the problem still remains.
> We observe the following behavior:
> 1. data is ingested at an extreme fast pace into memtables and flush queue 
> fills up
> 2. the available memory limit for memtables is reached and writes are no 
> longer accepted
> 3. the application gets hit by "write timeout", and retries repeatedly, in 
> vain 
> 4. after several failed attempts to write, the job gets aborted 
> Desired behaviour:
> 1. data is ingested at an extreme fast pace into memtables and flush queue 
> fills up
> 2. after exceeding some memtable "fill threshold", C* applies adaptive rate 
> limiting to writes - the more the buffers are filled-up, the less writes/s 
> are accepted, however writes still occur within the write timeout.
> 3. thanks to slowed down data ingestion, now flush can finish before all the 
> memory gets used
> Of course the details how rate limiting could be done are up for a discussion.
> It may be also worth considering putting such logic into the driver, not C* 
> core, but then C* needs to expose at least the following information to the 
> driver, so we could calculate the desired maximum data rate:
> 1. current amount of memory available for writes before they would completely 
> block
> 2. total amount of data queued to be flushed and flush progress (amount of 
> data to flush remaining for the memtable currently being flushed)
> 3. average flush write speed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12056) dtest failure in pushed_notifications_test.TestPushedNotifications.add_and_remove_node_test

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344959#comment-15344959
 ] 

Sam Tunnicliffe commented on CASSANDRA-12056:
-

The test in question was added for CASSANDRA-11038, which went into 2.2+. I've 
opened a [trivial dtest 
PR|https://github.com/riptano/cassandra-dtest/pull/1053] to exclude it for for 
test runs on earlier versions.

> dtest failure in 
> pushed_notifications_test.TestPushedNotifications.add_and_remove_node_test
> ---
>
> Key: CASSANDRA-12056
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12056
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node2.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/482/testReport/pushed_notifications_test/TestPushedNotifications/add_and_remove_node_test
> Failed on CassCI build cassandra-2.1_dtest #482
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/pushed_notifications_test.py", line 
> 242, in add_and_remove_node_test
> self.assertEquals(2, len(notifications), notifications)
>   File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
> assertion_func(first, second, msg=msg)
>   File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
> raise self.failureException(msg)
> "[{'change_type': u'NEW_NODE', 'address': ('127.0.0.2', 9042)}]
> {code}
> Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

[Sergio Bossa (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344958#comment-15344958
 ] 

Sergio Bossa commented on CASSANDRA-9318:
-

I would like to reopen this ticket and propose the following patch to implement 
coordinator-based back-pressure:

| [3.0 
patch|https://github.com/apache/cassandra/compare/cassandra-3.0...sbtourist:CASSANDRA-9318-3.0?expand=1]
 | 
[testall|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-3.0-testall/]
 | 
[dtest|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-3.0-dtest/]
 |

The above patch provides full-blown, end-to-end, 
replica-to-coordinator-to-client *write* back-pressure, based on the following 
main concepts:
* The replica itself has no back-pressure knowledge: it keeps trying to write 
mutations as fast as possible, and still applies load shedding.
* The coordinator tracks the back-pressure state *per replica*, which in the 
current implementation consists of the incoming and outgoing rate of messages 
from/to the replica.
* The coordinator is configured with a back-pressure strategy that based on the 
back-pressure state, applies a given back-pressure algorithm when sending 
mutations to each replica.
* The provided default strategy is based on the incoming/outgoing message 
ratio, used to rate limit outgoing messages towards a given replica.
* The back-pressure strategy is also in charge of signalling the coordinator 
when a given replica is considered "overloaded", in which case an 
{{OverloadedException}} is thrown to the client for all mutation requests 
deemed as "overloading", until the strategy considers such overloaded state 
over.
* The provided default strategy uses configurable low/high thresholds to either 
rate limit or throw exception back to clients.

While all of that might seem too complex, the patch is actually surprisingly 
simple. I provided as many unit tests as possible, and I've also tested it on a 
2-nodes CCM cluster, using [ByteMan|http://byteman.jboss.org/] to simulate a 
slow replica, and I'd say results are quite promising: as an example, see 
attached ByteMan script and plots showing a cluster with no back-pressure 
ending up dropping ~200k mutations, while a cluster with back-pressure enabled 
only ~2k, which means less coordinator overload and an easily recoverable 
replica state via hints.

I can foresee at least two open points:
* We might want to track more "back-pressure state" to allow implementing 
different strategies; I personally believe strategies based on in/out rates are 
the most appropriate ones to avoid *both* the overloading and dropped mutations 
problems, but people might think differently.
* When the {{OverloadedException}} is (eventually) thrown, some requests might 
have been already sent, which is exactly what currently happens with hint 
overloading too: we might want to check both kinds of overloading before 
actually sending any mutations to replicas.

Thoughts?

> Bound the number of in-flight requests at the coordinator
> -
>
> Key: CASSANDRA-9318
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths, Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> It's possible to somewhat bound the amount of load accepted into the cluster 
> by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding 
> bytes and requests and if it reaches a high watermark disable read on client 
> connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't 
> introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12069) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_options_from_config_file

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12069:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_options_from_config_file
 Key: CASSANDRA-12069
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12069
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_offheap_dtest/262/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_copy_options_from_config_file

Failed on CassCI build trunk_offheap_dtest #262

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2203, in test_copy_options_from_config_file
[('header', 'True'), ('maxattempts', '9')])
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2194, in do_test
check_options(out, expected_options)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2176, in check_options
d = json.loads(opts)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
'No JSON object could be decoded
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12068) dtest failure in paging_test.TestPagingData.static_columns_paging_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12068:
-

 Summary: dtest failure in 
paging_test.TestPagingData.static_columns_paging_test
 Key: CASSANDRA-12068
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12068
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log

example failure:

http://cassci.datastax.com/job/trunk_offheap_dtest/261/testReport/paging_test/TestPagingData/static_columns_paging_test

Failed on CassCI build trunk_offheap_dtest #261

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/paging_test.py", line 715, in 
static_columns_paging_test
self.assertEqual(16, len(results))
  File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
raise self.failureException(msg)
"16 != 6
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12064) dtest failure in auth_test.TestAuth.login_test

[Sam Tunnicliffe (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344929#comment-15344929
 ] 

Sam Tunnicliffe commented on CASSANDRA-12064:
-

I think we need some additional logging in the dtest to diagnose this. 

All we know is that the client was not able to connect when it supplied the bad 
password, but that the exception was not {{AuthenticationFailed}}. The server 
was operational at that point as the connection with the correct password had 
already succeeded. There's nothing in the server logs to indicate what the 
problem might be and I don't see any huge pauses in the GC log, so my only 
suggestion is to have the dtest output which exception it hits & check if the 
test fails again (though this particular test is very old and hasn't failed in 
around forever).

> dtest failure in auth_test.TestAuth.login_test
> --
>
> Key: CASSANDRA-12064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12064
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1284/testReport/auth_test/TestAuth/login_test
> Failed on CassCI build trunk_dtest #1284
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/auth_test.py", line 82, in login_test
> assert isinstance(e.errors.values()[0], AuthenticationFailed)
> {code}
> Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12067) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_explicit_column_order_reading

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12067:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_explicit_column_order_reading
 Key: CASSANDRA-12067
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12067
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_novnode_dtest/405/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_explicit_column_order_reading

Failed on CassCI build trunk_novnode_dtest #405

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
1453, in test_explicit_column_order_reading
self.assertCsvResultEqual(reference_file.name, results, 'testorder')
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
318, in assertCsvResultEqual
raise e
"Element counts were not equal:\nFirst has 1, Second has 0:  ['1', 'ham', 
'20']\nFirst has 1, Second has 0:  ['2', 'eggs', '40']\nFirst has 1, Second has 
0:  ['3', 'beans', '60']\nFirst has 1, Second has 0:  ['4', 'toast', '80']
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11381) Node running with join_ring=false and authentication can not serve requests

[Joel Knighton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Knighton updated CASSANDRA-11381:
--
Status: Open  (was: Patch Available)

> Node running with join_ring=false and authentication can not serve requests
> ---
>
> Key: CASSANDRA-11381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11381
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: 11381-2.0.txt, 11381-2.1.txt, 11381-2.2.txt, 
> 11381-3.0.txt, 11381-trunk.txt, dtest-11381-trunk.txt
>
>
> Starting up a node with {{-Dcassandra.join_ring=false}} in a cluster that has 
> authentication configured, eg PasswordAuthenticator, won't be able to serve 
> requests. This is because {{Auth.setup()}} never gets called during the 
> startup.
> Without {{Auth.setup()}} having been called in {{StorageService}} clients 
> connecting to the node fail with the node throwing
> {noformat}
> java.lang.NullPointerException
> at 
> org.apache.cassandra.auth.PasswordAuthenticator.authenticate(PasswordAuthenticator.java:119)
> at 
> org.apache.cassandra.thrift.CassandraServer.login(CassandraServer.java:1471)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3505)
> at 
> org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3489)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at com.thinkaurelius.thrift.Message.invoke(Message.java:314)
> at 
> com.thinkaurelius.thrift.Message$Invocation.execute(Message.java:90)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:695)
> at 
> com.thinkaurelius.thrift.TDisruptorServer$InvocationHandler.onEvent(TDisruptorServer.java:689)
> at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:112)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The exception thrown from the 
> [code|https://github.com/apache/cassandra/blob/cassandra-2.0.16/src/java/org/apache/cassandra/auth/PasswordAuthenticator.java#L119]
> {code}
> ResultMessage.Rows rows = 
> authenticateStatement.execute(QueryState.forInternalCalls(), new 
> QueryOptions(consistencyForUser(username),
>   
>Lists.newArrayList(ByteBufferUtil.bytes(username;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12066) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_different_number_precision

[Sean McCarthy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean McCarthy resolved CASSANDRA-12066.
---
Resolution: Duplicate

Duplicate of #12061

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_different_number_precision
> ---
>
> Key: CASSANDRA-12066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12066
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Attachments: node1.log, node1_debug.log, node1_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_novnode_dtest/405/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_round_trip_with_different_number_precision
> Failed on CassCI build trunk_novnode_dtest #405
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
> f(obj)
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 2049, in test_round_trip_with_different_number_precision
> do_test(0, 0)
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 2044, in do_test
> self.assertItemsEqual(sorted(list(csv_rows(tempfile1.name))), 
> sorted(list(csv_rows(tempfile2.name
>   File "/usr/lib/python2.7/unittest/case.py", line 901, in assertItemsEqual
> self.fail(msg)
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> "Element counts were not equal:\nFirst has 1, Second has 0:  ['1', '1', '1']
> {code}
> Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12066) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_different_number_precision

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12066:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_different_number_precision
 Key: CASSANDRA-12066
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12066
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_novnode_dtest/405/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_round_trip_with_different_number_precision

Failed on CassCI build trunk_novnode_dtest #405

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2049, in test_round_trip_with_different_number_precision
do_test(0, 0)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2044, in do_test
self.assertItemsEqual(sorted(list(csv_rows(tempfile1.name))), 
sorted(list(csv_rows(tempfile2.name
  File "/usr/lib/python2.7/unittest/case.py", line 901, in assertItemsEqual
self.fail(msg)
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)
"Element counts were not equal:\nFirst has 1, Second has 0:  ['1', '1', '1']
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: ninja: fix logic to log msg w/cdc check on startup

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 09103eeac -> 82ef1cfc3


ninja: fix logic to log msg w/cdc check on startup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/82ef1cfc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/82ef1cfc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/82ef1cfc

Branch: refs/heads/trunk
Commit: 82ef1cfc3d5a62c4d0f6163e09a9690b43959a92
Parents: 09103ee
Author: Josh McKenzie 
Authored: Wed Jun 22 13:43:55 2016 -0400
Committer: Josh McKenzie 
Committed: Wed Jun 22 13:43:55 2016 -0400

--
 src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/82ef1cfc/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 5b3e57d..e17a2bc 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -593,7 +593,7 @@ public class DatabaseDescriptor
 }
 }
 
-if (conf.cdc_enabled != null)
+if (conf.cdc_enabled == true)
 {
 logger.info("cdc_enabled is true. Starting casssandra node with 
Change-Data-Capture enabled.");
 }

[jira] [Updated] (CASSANDRA-12026) update NEWS.txt to explain system schema exceptions during partial cluster upgrade

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12026:

Resolution: Fixed
Status: Resolved  (was: Ready to Commit)

committed. thanks.

> update NEWS.txt to explain system schema exceptions during partial cluster 
> upgrade
> --
>
> Key: CASSANDRA-12026
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12026
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Joshua McKenzie
> Attachments: 12026_v1.txt
>
>
> Upgrade tests found this exception occuring during upgrades:
> {noformat}
> node2: ERROR [MessagingService-Incoming-/127.0.0.1] 2016-06-16 20:14:59,268 
> CassandraDaemon.java:217 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.RuntimeException: Unknown column cdc during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:656)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:341)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:350)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:610)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:593)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.7.jar:3.7]
> {noformat}
> Which is apparently normal and should subside after full cluster upgrade to 
> post-3.0 versions.
> NEWS.txt needs an update to let users know this is not a problem during their 
> upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12026) update NEWS.txt to explain system schema exceptions during partial cluster upgrade

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-12026:

Fix Version/s: 3.8

> update NEWS.txt to explain system schema exceptions during partial cluster 
> upgrade
> --
>
> Key: CASSANDRA-12026
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12026
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Russ Hatch
>Assignee: Joshua McKenzie
> Fix For: 3.8
>
> Attachments: 12026_v1.txt
>
>
> Upgrade tests found this exception occuring during upgrades:
> {noformat}
> node2: ERROR [MessagingService-Incoming-/127.0.0.1] 2016-06-16 20:14:59,268 
> CassandraDaemon.java:217 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.lang.RuntimeException: Unknown column cdc during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:656)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:341)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:350)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:610)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:593)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[apache-cassandra-3.7.jar:3.7]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[apache-cassandra-3.7.jar:3.7]
> {noformat}
> Which is apparently normal and should subside after full cluster upgrade to 
> post-3.0 versions.
> NEWS.txt needs an update to let users know this is not a problem during their 
> upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Explain errors during schema migration on mixed-version clusters post 3.0

[jmckenzie]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4ae198765 -> 09103eeac


Explain errors during schema migration on mixed-version clusters post 3.0

Patch by jmckenzie; reviewed by cyeksigian for CASSANDRA-12026


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/09103eea
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/09103eea
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/09103eea

Branch: refs/heads/trunk
Commit: 09103eeac93291a193db9a39b8a68964aacfe4d2
Parents: 4ae1987
Author: Josh McKenzie 
Authored: Wed Jun 22 13:24:40 2016 -0400
Committer: Josh McKenzie 
Committed: Wed Jun 22 13:24:40 2016 -0400

--
 NEWS.txt | 7 +++
 1 file changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/09103eea/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index d34613c..7418f3a 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -48,6 +48,13 @@ Upgrading
   those under a different name, change your code to use the new names and
   drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 
for more
   details).
+- Due to changes in schema migration handling and the storage format after 
3.0, you will
+  see error messages such as:
+ "java.lang.RuntimeException: Unknown column cdc during 
deserialization"
+  in your system logs on a mixed-version cluster during upgrades. This 
error message
+  is harmless and due to the 3.8 nodes having cdc added to their schema 
tables while
+  the <3.8 nodes do not. This message should cease once all nodes are 
upgraded to 3.8.
+  As always, refrain from schema changes during cluster upgrades.
 
 Deprecation
 ---

[jira] [Updated] (CASSANDRA-12007) dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test

[Sean McCarthy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean McCarthy updated CASSANDRA-12007:
--
Reviewer: Tyler Hobbs
  Status: Patch Available  (was: In Progress)

https://github.com/riptano/cassandra-dtest/pull/1052

> dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test
> 
>
> Key: CASSANDRA-12007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12007
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Sean McCarthy
>  Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/229/testReport/cql_tracing_test/TestCqlTracing/tracing_simple_test
> Failed on CassCI build cassandra-2.1_dtest_jdk8 #229
> Logs are attached.
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 104, in 
> tracing_simple_test
> self.trace(session)
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 92, in 
> trace
> self.assertIn(' 127.0.0.2 ', out)
>   File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
> self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> "' 127.0.0.2 ' not found
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12065) dtest failure in repair_tests.repair_test.TestRepair.repair_after_upgrade_test

[Craig Kodman (JIRA)]

Craig Kodman created CASSANDRA-12065:


 Summary: dtest failure in 
repair_tests.repair_test.TestRepair.repair_after_upgrade_test
 Key: CASSANDRA-12065
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12065
 Project: Cassandra
  Issue Type: Test
Reporter: Craig Kodman
Assignee: DS Test Eng


example failure:

http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/259/testReport/repair_tests.repair_test/TestRepair/repair_after_upgrade_test

Failed on CassCI build cassandra-3.0_dtest_win32 #259



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12007) dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344749#comment-15344749
 ] 

Philip Thompson commented on CASSANDRA-12007:
-

I believe you are correct, we skipped the wrong portion of the test as a result 
of 11598. I do not think we should version gate the entire test.

> dtest failure in cql_tracing_test.TestCqlTracing.tracing_simple_test
> 
>
> Key: CASSANDRA-12007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12007
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sean McCarthy
>Assignee: Sean McCarthy
>  Labels: dtest
> Attachments: node1.log, node2.log, node3.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest_jdk8/229/testReport/cql_tracing_test/TestCqlTracing/tracing_simple_test
> Failed on CassCI build cassandra-2.1_dtest_jdk8 #229
> Logs are attached.
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 104, in 
> tracing_simple_test
> self.trace(session)
>   File "/home/automaton/cassandra-dtest/cql_tracing_test.py", line 92, in 
> trace
> self.assertIn(' 127.0.0.2 ', out)
>   File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
> self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> "' 127.0.0.2 ' not found
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12036) dtest failure in repair_tests.repair_test.TestRepair.repair_after_upgrade_test

[Philip Thompson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344739#comment-15344739
 ] 

Philip Thompson commented on CASSANDRA-12036:
-

Yes, that is possible. Pinged you privately

> dtest failure in repair_tests.repair_test.TestRepair.repair_after_upgrade_test
> --
>
> Key: CASSANDRA-12036
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12036
> Project: Cassandra
>  Issue Type: Test
>Reporter: Craig Kodman
>Assignee: DS Test Eng
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/257/testReport/repair_tests.repair_test/TestRepair/repair_after_upgrade_test
> Failed on CassCI build cassandra-3.0_dtest_win32 #257



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11993) Cannot read Snappy compressed tables with 3.6

[Nimi Wariboko Jr. (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344738#comment-15344738
 ] 

Nimi Wariboko Jr. commented on CASSANDRA-11993:
---

Copying my email from the dev mailing list...

Thanks for the explanation - I think it may be more likely I'm hitting 
MEMORY_USAGE_THRESHOLD (file_cache_size_in_mb, which is 512mb by default), as 
the table itself is very large (a couple TB over 12 nodes). In that case, due 
to the changes outlined CASSANDRA-9548/CASSANDRA-8709/CASSANDRA-5863, there 
should probably be a warning in 3.6 that buffer_pool_use_heap_if_exhausted 
should be set to false if you still have some tables on SnappyCompression.

> Cannot read Snappy compressed tables with 3.6
> -
>
> Key: CASSANDRA-11993
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11993
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Nimi Wariboko Jr.
> Fix For: 3.6
>
>
> After upgrading to 3.6, I can no longer read/compact sstables compressed with 
> snappy compression. The memtable_allocation_type makes no difference both 
> offheap_buffers and heap_buffers cause the errors.
> {code}
> WARN  [SharedPool-Worker-5] 2016-06-10 15:45:18,731 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-5,5,main]: {}
> org.xerial.snappy.SnappyError: [NOT_A_DIRECT_BUFFER] destination is not a 
> direct buffer
>   at org.xerial.snappy.Snappy.uncompress(Snappy.java:509) 
> ~[snappy-java-1.1.1.7.jar:na]
>   at 
> org.apache.cassandra.io.compress.SnappyCompressor.uncompress(SnappyCompressor.java:102)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.io.util.CompressedSegmentedFile$Mmap.readChunk(CompressedSegmentedFile.java:323)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:137) 
> ~[apache-cassandra-3.6.jar:3.6]
>   at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:19) 
> ~[apache-cassandra-3.6.jar:3.6]
>   at 
> com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:2949)
>  ~[caffeine-2.2.6.jar:na]
>   at 
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$15(BoundedLocalCache.java:1807)
>  ~[caffeine-2.2.6.jar:na]
>   at 
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853) 
> ~[na:1.8.0_66]
>   at 
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:1805)
>  ~[caffeine-2.2.6.jar:na]
>   at 
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:1788)
>  ~[caffeine-2.2.6.jar:na]
>   at 
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:97)
>  ~[caffeine-2.2.6.jar:na]
>   at 
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:66)
>  ~[caffeine-2.2.6.jar:na]
>   at 
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:215)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.cache.ChunkCache$CachingRebufferer.rebuffer(ChunkCache.java:193)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:78)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:220)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.io.util.SegmentedFile.createReader(SegmentedFile.java:138)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1779)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:44)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:72)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:65)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.initializeIterator(UnfilteredRowIteratorWithLowerBound.java:85)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
>  ~[apache-cassandra-3.6.jar:3.6]
>   at 
> 

[jira] [Commented] (CASSANDRA-12024) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_child_process_crashing

[Sean McCarthy (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344736#comment-15344736
 ] 

Sean McCarthy commented on CASSANDRA-12024:
---

[~stefania], I ran your patch: 
http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/133/testReport/

It seems a child process hits a range which should fail.

> dtest failure in 
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_copy_to_with_child_process_crashing
> 
>
> Key: CASSANDRA-12024
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12024
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Sean McCarthy
>  Labels: dtest
> Attachments: node1.log
>
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_offheap_dtest/360/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_copy_to_with_child_process_crashing
> Failed on CassCI build cassandra-2.1_offheap_dtest #360
> {code}
> Stacktrace
>   File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
>   File "/home/automaton/cassandra-dtest/dtest.py", line 889, in wrapped
> f(obj)
>   File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
> line 2701, in test_copy_to_with_child_process_crashing
> self.assertIn('some records might be missing', err)
>   File "/usr/lib/python2.7/unittest/case.py", line 803, in assertIn
> self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
> raise self.failureException(msg)
> Error Message
> 'some records might be missing' not found in ''
> {code}
> Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12045) Cassandra failure during write query at consistency LOCAL_QUORUM

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344669#comment-15344669
 ] 

Tyler Hobbs commented on CASSANDRA-12045:
-

What version of Cassandra are you using?

Your logback.xml file should specify the location of your system.log file.

>  Cassandra failure during write query at consistency LOCAL_QUORUM 
> --
>
> Key: CASSANDRA-12045
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12045
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL, Local Write-Read Paths
> Environment: Eclipse java environment
>Reporter: Raghavendra Pinninti
> Fix For: 3.x
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> While I am writing xml file into Cassandra table column I am facing following 
> exception.Its a 3 node cluster and All nodes are up.
> com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure 
> during write query at consistency LOCAL_QUORUM (2 responses were required but 
> only 0 replica responded, 1 failed) at 
> com.datastax.driver.core.exceptions.WriteFailureException.copy(WriteFailureException.java:80)
>  at 
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
>  at 
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
>  at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:55) 
> at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39) 
> at DBConnection.oracle2Cassandra(DBConnection.java:267) at 
> DBConnection.main(DBConnection.java:292) Caused by: 
> com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure 
> during write query at consistency LOCAL_QUORUM (2 responses were required but 
> only 0 replica responded, 1 failed) at 
> com.datastax.driver.core.exceptions.WriteFailureException.copy(WriteFailureException.java:91)
>  at com.datastax.driver.core.Responses$Error.asException(Responses.java:119) 
> at 
> com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:180)
>  at 
> com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:186)
>  at 
> com.datastax.driver.core.RequestHandler.access$2300(RequestHandler.java:44) 
> at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:754)
>  at 
> com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:576)
> It would be great if someone helps me out from this situation. Thanks
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12054) Add CQL Data Modeler to tree

[Joshua McKenzie (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344653#comment-15344653
 ] 

Joshua McKenzie commented on CASSANDRA-12054:
-

Current stress is badly in need of some user-friendliness so I'd counsel 
against making this ticket contingent on anything other than a minimum viable 
to get it in-tree. We should do 2,5,7, and potential for running stress and 
storing history as separate efforts.

> Add CQL Data Modeler to tree
> 
>
> Key: CASSANDRA-12054
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12054
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Sebastian Estevez
>
> cassandra-stess, like many powerful tools, is not easy to use. It requires 
> some statistical understanding and syntactic skill, in order to get up and 
> running with even a simple data model, user profile driven test (for details 
> see the cassandra-stress docs). Furthermore, designing a good cassandra data 
> model requires basic understanding of how CQL works and how c* data is laid 
> out on disk as a result of partitioning and clustering.
> The CQL DataModeler aims to simplify this task, getting users up and running 
> with user profile powered cassandra-stress tests in minutes.
> Given the feedback the community has voiced about the usability of 
> cassandra-stress at NGCC, it was suggested that I contribute the data modeler 
> to the open source project so it can be maintained in tree and leveraged by 
> users.
> It is a simple static web application and users should be able to use it by 
> just opening up index.html with their browser and populating the GUI.
> Check it out here:
> http://www.sestevez.com/sestevez/CassandraDataModeler/
> The source code sits in github, once we clean it up and know where it will 
> live in tree, I'll submit a c* patch:
> https://github.com/phact/CassandraDataModeler
> I have developed this as a side project and not as production ready code. I 
> welcome feedback on how it can be cleaned up and improved.
> cc: [~tjake][~carlyeks]
> Future improvements include:
> 1) Add cluster distributions (currently only size and population are 
> supported)
> 2) Add functionality so that the histograms display overall distributions 
> (combining cluster and population distributions for fields)
> 3) Include batch configuration and insert distribution
> 4) Include -pop and other command line options that are crucial for 
> describing workloads
> 5) Add sparse table capabilities (already in stress but currently 
> undocumented)
> 6) Add a few example data models to ship with the tool
> 7) Eventually allow users to contribute back profiles to some sort of 
> community
> IMO this jira should be contingent on 1, 3, 4, and 6 being completed. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-11937:

Status: Patch Available  (was: Open)

Multiplexed test has passed 30 runs without OOM (sorry for format there, I'm 
still unsure how to launch jobs that would produce sensible output, red button 
there only indicates the final SSH failure, not test failure, so I had to check 
output manually). Same result locally.

|[trunk|https://github.com/ifesdjeen/cassandra/tree/11937-trunk]|[utest|http://cassci.datastax.com/job/ifesdjeen-11937-trunk-testall/1/]|[dtest|http://cassci.datastax.com/job/ifesdjeen-11937-trunk-dtest/1/]|[multiplexed
 tests|http://cassci.datastax.com/job/ifesdjeen-11937-trunk-multiplexed/1/]|

Test failures seem unrelated and have same result on Trunk for me...

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12064) dtest failure in auth_test.TestAuth.login_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12064:
-

 Summary: dtest failure in auth_test.TestAuth.login_test
 Key: CASSANDRA-12064
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12064
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1284/testReport/auth_test/TestAuth/login_test

Failed on CassCI build trunk_dtest #1284

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/auth_test.py", line 82, in login_test
assert isinstance(e.errors.values()[0], AuthenticationFailed)
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12063) Brotli storage compression

[Eric Evans (JIRA)]

Eric Evans created CASSANDRA-12063:
--

 Summary: Brotli storage compression
 Key: CASSANDRA-12063
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12063
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Eric Evans
Assignee: Eric Evans
Priority: Minor


Brotli is a compression algorithm based on a modern variant of the LZ77 
algorithm, Huffman coding and 2nd order context modeling.  It produces smaller 
compressed sizes at costs comparable to deflate.

I have a working [ICompressor 
implementation|https://github.com/eevans/cassandra-brotli] which has received a 
fair amount of testing already. I'll follow up shortly with a Cassandra 
changeset(s) for review.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12062) Prevent key invalidation if there's no key to invalidate

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-12062:
-
Fix Version/s: 3.x
   Status: Patch Available  (was: Open)

Patch available (just started CI)
||trunk|[branch|https://github.com/apache/cassandra/compare/trunk...snazy:12062-invalidate-opts-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12062-invalidate-opts-trunk-testall/lastSuccessfulBuild/]|[dtest|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-12062-invalidate-opts-trunk-dtest/lastSuccessfulBuild/]


> Prevent key invalidation if there's no key to invalidate
> 
>
> Key: CASSANDRA-12062
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12062
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.x
>
>
> In {{SSTableRewriter}} we currently assume that there are keys to be 
> invalidated in the key cache. That's not always true - especially if a bunch 
> of tiny sstables have been generated.
> Patch saves one {{synchronized}} and a few objects if that's the case.
> Just a small optimisation, but maybe worth to include.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12062) Prevent key invalidation if there's no key to invalidate

[Robert Stupp (JIRA)]

Robert Stupp created CASSANDRA-12062:


 Summary: Prevent key invalidation if there's no key to invalidate
 Key: CASSANDRA-12062
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12062
 Project: Cassandra
  Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Minor


In {{SSTableRewriter}} we currently assume that there are keys to be 
invalidated in the key cache. That's not always true - especially if a bunch of 
tiny sstables have been generated.

Patch saves one {{synchronized}} and a few objects if that's the case.

Just a small optimisation, but maybe worth to include.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12054) Add CQL Data Modeler to tree

[Jeremy Hanna (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344609#comment-15344609
 ] 

Jeremy Hanna commented on CASSANDRA-12054:
--

As we talked about offline, it would be great to have a way to plug this in so 
it could actually run stress, store the history including different profiles, 
and integrate the new graphing tool for stress from 
CASSANDRA-9870/CASSANDRA-7918.

> Add CQL Data Modeler to tree
> 
>
> Key: CASSANDRA-12054
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12054
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Sebastian Estevez
>
> cassandra-stess, like many powerful tools, is not easy to use. It requires 
> some statistical understanding and syntactic skill, in order to get up and 
> running with even a simple data model, user profile driven test (for details 
> see the cassandra-stress docs). Furthermore, designing a good cassandra data 
> model requires basic understanding of how CQL works and how c* data is laid 
> out on disk as a result of partitioning and clustering.
> The CQL DataModeler aims to simplify this task, getting users up and running 
> with user profile powered cassandra-stress tests in minutes.
> Given the feedback the community has voiced about the usability of 
> cassandra-stress at NGCC, it was suggested that I contribute the data modeler 
> to the open source project so it can be maintained in tree and leveraged by 
> users.
> It is a simple static web application and users should be able to use it by 
> just opening up index.html with their browser and populating the GUI.
> Check it out here:
> http://www.sestevez.com/sestevez/CassandraDataModeler/
> The source code sits in github, once we clean it up and know where it will 
> live in tree, I'll submit a c* patch:
> https://github.com/phact/CassandraDataModeler
> I have developed this as a side project and not as production ready code. I 
> welcome feedback on how it can be cleaned up and improved.
> cc: [~tjake][~carlyeks]
> Future improvements include:
> 1) Add cluster distributions (currently only size and population are 
> supported)
> 2) Add functionality so that the histograms display overall distributions 
> (combining cluster and population distributions for fields)
> 3) Include batch configuration and insert distribution
> 4) Include -pop and other command line options that are crucial for 
> describing workloads
> 5) Add sparse table capabilities (already in stress but currently 
> undocumented)
> 6) Add a few example data models to ship with the tool
> 7) Eventually allow users to contribute back profiles to some sort of 
> community
> IMO this jira should be contingent on 1, 3, 4, and 6 being completed. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12061) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_different_number_precision

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12061:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_round_trip_with_different_number_precision
 Key: CASSANDRA-12061
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12061
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1284/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_round_trip_with_different_number_precision

Failed on CassCI build trunk_dtest #1284

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2046, in test_round_trip_with_different_number_precision
do_test(None, None)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
2044, in do_test
self.assertItemsEqual(sorted(list(csv_rows(tempfile1.name))), 
sorted(list(csv_rows(tempfile2.name
  File "/usr/lib/python2.7/unittest/case.py", line 901, in assertItemsEqual
self.fail(msg)
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)
"Element counts were not equal:\nFirst has 1, Second has 0:  ['1', '1.1235', 
'1.12345678912']
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12060) Different failure format for failed LWT between 2.x and 3.x

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-12060:

Description: 
When executing following CQL commands: 

{code}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1': '1' };
USE test;
CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
PRIMARY KEY (a, b));
INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
{code}

The output is different between {{2.x}} and {{3.x}}:

2.x:
{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied] | s2
---+--
 False | null
{code}

3.x:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied]
---
 False
{code}

{{2.x}} would although return same result if executed on a partition that does 
not exist at all:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;

 [applied]
---
 False
{code}

It _might_ be related to static column LWTs, as I could not reproduce same 
behaviour with non-static column LWTs. The most recent change was 
[CASSANDRA-10532], which enabled LWT operations on static columns with 
partition keys only. Another possible relation is [CASSANDRA-9842], which 
removed distinction between {{null}} column and non-existing row.

  was:
When executing following CQL commands: 

{code}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1': '1' };
USE test;
CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
PRIMARY KEY (a, b));
INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
{code}

The output is different between {{2.x}} and {{3.x}}:

2.x:
{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied] | s2
---+--
 False | null
{code}

3.x:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied]
---
 False
{code}

{{2.x}} would although return same result if executed on a partition that does 
not exist at all:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;

 [applied]
---
 False
{code}

It _might_ be related to static column LWTs, as I could not reproduce same 
behaviour with non-static column LWTs. The most recent change was 
[CASSANDRA-10532], which enabled LWT operations on static columns with 
partition keys only. Another possible relation is [CASSANDRA-9842], which 
removed distinction between {{null}} and non-existing rows.


> Different failure format for failed LWT between 2.x and 3.x
> ---
>
> Key: CASSANDRA-12060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12060
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Petrov
>
> When executing following CQL commands: 
> {code}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1' };
> USE test;
> CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
> PRIMARY KEY (a, b));
> INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
> DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
> {code}
> The output is different between {{2.x}} and {{3.x}}:
> 2.x:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;
>  [applied] | s2
> ---+--
>  False | null
> {code}
> 3.x:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;
>  [applied]
> ---
>  False
> {code}
> {{2.x}} would although return same result if executed on a partition that 
> does not exist at all:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;
>  [applied]
> ---
>  False
> {code}
> It _might_ be related to static column LWTs, as I could not reproduce same 
> behaviour with non-static column LWTs. The most recent change was 
> [CASSANDRA-10532], which enabled LWT operations on static columns with 
> partition keys only. Another possible relation is [CASSANDRA-9842], which 
> removed distinction between {{null}} column and non-existing row.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12060) Different failure format for failed LWT between 2.x and 3.x

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-12060:

Description: 
When executing following CQL commands: 

{code}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1': '1' };
USE test;
CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
PRIMARY KEY (a, b));
INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
{code}

The output is different between {{2.x}} and {{3.x}}:

2.x:
{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied] | s2
---+--
 False | null
{code}

3.x:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied]
---
 False
{code}

{{2.x}} would although return same result if executed on a partition that does 
not exist at all:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;

 [applied]
---
 False
{code}

It _might_ be related to static column LWTs, as I could not reproduce same 
behaviour with non-static column LWTs. The most recent change was 
[CASSANDRA-10532], which enabled LWT operations on static columns with 
partition keys only. Another possible relation is [CASSANDRA-9842], which 
removed distinction between {{null}} and non-existing rows.

  was:
When executing following CQL commands: 

{code}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1': '1' };
USE test;
CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
PRIMARY KEY (a, b));
INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
{code}

The output is different between {{2.x}} and {{3.x}}:

2.x:
{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied] | s2
---+--
 False | null
{code}

3.x:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied]
---
 False
{code}

{{2.x}} would although return same result if executed on a partition that does 
not exist at all:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;

 [applied]
---
 False
{code}

It _might_ be related to static column LWTs, as I could not reproduce same 
behaviour with non-static column LWTs. The most recent change was 
[CASSANDRA-10532].



> Different failure format for failed LWT between 2.x and 3.x
> ---
>
> Key: CASSANDRA-12060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12060
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Petrov
>
> When executing following CQL commands: 
> {code}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1' };
> USE test;
> CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
> PRIMARY KEY (a, b));
> INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
> DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
> {code}
> The output is different between {{2.x}} and {{3.x}}:
> 2.x:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;
>  [applied] | s2
> ---+--
>  False | null
> {code}
> 3.x:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;
>  [applied]
> ---
>  False
> {code}
> {{2.x}} would although return same result if executed on a partition that 
> does not exist at all:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;
>  [applied]
> ---
>  False
> {code}
> It _might_ be related to static column LWTs, as I could not reproduce same 
> behaviour with non-static column LWTs. The most recent change was 
> [CASSANDRA-10532], which enabled LWT operations on static columns with 
> partition keys only. Another possible relation is [CASSANDRA-9842], which 
> removed distinction between {{null}} and non-existing rows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12060) Different failure format for failed LWT between 2.x and 3.x

[Alex Petrov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-12060:

Description: 
When executing following CQL commands: 

{code}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1': '1' };
USE test;
CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
PRIMARY KEY (a, b));
INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
{code}

The output is different between {{2.x}} and {{3.x}}:

2.x:
{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied] | s2
---+--
 False | null
{code}

3.x:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied]
---
 False
{code}

{{2.x}} would although return same result if executed on a partition that does 
not exist at all:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;

 [applied]
---
 False
{code}

It _might_ be related to static column LWTs, as I could not reproduce same 
behaviour with non-static column LWTs. The most recent change was 
[CASSANDRA-10532].


  was:
When executing following CQL commands: 

{code}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1': '1' };
USE test;
CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
PRIMARY KEY (a, b));
INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
{code}

The output is different between {{2.x}} and {{3.x}}:

2.x:
{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied] | s2
---+--
 False | null
{code}

3.x:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied]
---
 False
{code}

{{2.x}} would although return same result if executed on a partition that does 
not exist at all:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;

 [applied]
---
 False
{code}


> Different failure format for failed LWT between 2.x and 3.x
> ---
>
> Key: CASSANDRA-12060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12060
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Alex Petrov
>
> When executing following CQL commands: 
> {code}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1' };
> USE test;
> CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
> PRIMARY KEY (a, b));
> INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
> DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
> {code}
> The output is different between {{2.x}} and {{3.x}}:
> 2.x:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;
>  [applied] | s2
> ---+--
>  False | null
> {code}
> 3.x:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;
>  [applied]
> ---
>  False
> {code}
> {{2.x}} would although return same result if executed on a partition that 
> does not exist at all:
> {code}
> cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;
>  [applied]
> ---
>  False
> {code}
> It _might_ be related to static column LWTs, as I could not reproduce same 
> behaviour with non-static column LWTs. The most recent change was 
> [CASSANDRA-10532].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12060) Different failure format for failed LWT between 2.x and 3.x

[Alex Petrov (JIRA)]

Alex Petrov created CASSANDRA-12060:
---

 Summary: Different failure format for failed LWT between 2.x and 
3.x
 Key: CASSANDRA-12060
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12060
 Project: Cassandra
  Issue Type: Bug
Reporter: Alex Petrov


When executing following CQL commands: 

{code}
CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
'datacenter1': '1' };
USE test;
CREATE TABLE testtable (a int, b int, s1 int static, s2 int static, v int, 
PRIMARY KEY (a, b));
INSERT INTO testtable (a,b,s1,s2,v) VALUES (2,2,2,null,2);
DELETE s1 FROM testtable WHERE a = 2 IF s2 IN (10,20,30);
{code}

The output is different between {{2.x}} and {{3.x}}:

2.x:
{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied] | s2
---+--
 False | null
{code}

3.x:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 2 IF s2 = 5;

 [applied]
---
 False
{code}

{{2.x}} would although return same result if executed on a partition that does 
not exist at all:

{code}
cqlsh:test> DELETE s1 FROM testtable WHERE a = 5 IF s2 = 5;

 [applied]
---
 False
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-3486) Node Tool command to stop repair

[Stefan Podkowinski (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344556#comment-15344556
 ] 

Stefan Podkowinski commented on CASSANDRA-3486:
---

I've now looked at this issue as a potential use case for CASSANDRA-12016 and 
added a test on top of it. The branch can be found at 
[WIP-3486|https://github.com/spodkowinski/cassandra/tree/WIP-3486] and the test 
I'm talking about in 
[ActiveRepairServiceMessagingTest.java|https://github.com/spodkowinski/cassandra/blob/3a9ba2edcfe5a3a774089884d5fa7f4df4c9b70c/test/unit/org/apache/cassandra/service/ActiveRepairServiceMessagingTest.java].
 

My goal was to be able to make coordination between different nodes in repair 
scenarios easier to test. The basic cases covered so far are pretty simple, but 
I like to add more edge cases in the future. Nonetheless I wanted to share this 
early on in case [~pauloricardomg] and others have any feedback on how this 
approach would be helpful to make progress on this issue.



> Node Tool command to stop repair
> 
>
> Key: CASSANDRA-3486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3486
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
> Environment: JVM
>Reporter: Vijay
>Assignee: Paulo Motta
>Priority: Minor
>  Labels: repair
> Fix For: 2.1.x
>
> Attachments: 0001-stop-repair-3583.patch
>
>
> After CASSANDRA-1740, If the validation compaction is stopped then the repair 
> will hang. This ticket will allow users to kill the original repair.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12059) dtest failure in cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_ttl

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12059:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_reading_with_ttl
 Key: CASSANDRA-12059
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12059
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1283/testReport/cqlsh_tests.cqlsh_copy_tests/CqlshCopyTest/test_reading_with_ttl

Failed on CassCI build trunk_dtest #1283

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
850, in test_reading_with_ttl
self.assertItemsEqual(data, result)
  File "/usr/lib/python2.7/unittest/case.py", line 901, in assertItemsEqual
self.fail(msg)
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)
"Element counts were not equal:\nFirst has 1, Second has 0:  [1, 20]\nFirst has 
1, Second has 0:  [2, 40]
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11904) Exception in thread Thread[CompactionExecutor:13358,1,main] java.lang.AssertionError: Memory was freed

[Dan Kinder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Kinder updated CASSANDRA-11904:
---
Attachment: system.log.2016-06-10_0514

Attached a log, search for "Memory was freed"

Note that this was on upgrade from 2.2.5 to 3.6. I can't do any further 
experimentation now because we had too many issues and gave up on this upgrade 
and stuck with 2.2.6. (Unfortunate, because we really wanted the jbod 
improvements introduced in 3.2.)

> Exception in thread Thread[CompactionExecutor:13358,1,main] 
> java.lang.AssertionError: Memory was freed
> --
>
> Key: CASSANDRA-11904
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11904
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Valentin Martinjuk
>Assignee: Marcus Eriksson
> Attachments: system.log.2016-06-10_0514
>
>
> We have Cassandra cluster 2.2.5 with two datacenters(3 nodes each).
> We observe ERRORs below on all nodes. The ERROR is repeated every minute. 
> No any complains from customers. Do we have any chance to fix it without 
> restart?
> {code}
> ERROR [CompactionExecutor:13996] 2016-05-26 21:20:46,700 
> CassandraDaemon.java:185 - Exception in thread 
> Thread[CompactionExecutor:13996,1,main]
> java.lang.AssertionError: Memory was freed
> at 
> org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103) 
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at org.apache.cassandra.io.util.Memory.getInt(Memory.java:292) 
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:148)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.IndexSummary.fillTemporaryKey(IndexSummary.java:162)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:121)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.format.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:1398)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.format.SSTableReader.estimatedKeysForRanges(SSTableReader.java:1354)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:403)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.findDroppableSSTable(LeveledCompactionStrategy.java:412)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:101)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:88)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:250)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> ~[na:1.8.0_74]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> ~[na:1.8.0_74]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_74]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_74]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
> ERROR [CompactionExecutor:13996] 2016-05-26 21:21:46,702 
> CassandraDaemon.java:185 - Exception in thread 
> Thread[CompactionExecutor:13996,1,main]
> java.lang.AssertionError: Memory was freed
> at 
> org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103) 
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at org.apache.cassandra.io.util.Memory.getInt(Memory.java:292) 
> ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:148)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.IndexSummary.fillTemporaryKey(IndexSummary.java:162)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:121)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> org.apache.cassandra.io.sstable.format.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:1398)
>  ~[apache-cassandra-2.2.5.jar:2.2.5]
> at 
> 

[jira] [Commented] (CASSANDRA-11368) Lists inserts are not truly idempotent

[Ngoc Minh Vo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344546#comment-15344546
 ] 

Ngoc Minh Vo commented on CASSANDRA-11368:
--

Hello,

We encounter the same issue randomly on our production setup with two 
datacenters using C* v2.0.14. No problem on one-datacenter UAT environment.

I believe that coordinator node sends insert query to two datacenters with the 
same timestamp. Hence, data spreading process inter-datacenter could lead to 
duplicated values in list.

Two possible solutions:
1. use "map" with keys corresponding to the value indexes (i.e.: 0, 
1, 2, ... N)
2. use blob to store all list values and perform deserialization on client side 
(via DAO layer for instance)

Any other recommendation?
Thanks for your help.
Minh

> Lists inserts are not truly idempotent
> --
>
> Key: CASSANDRA-11368
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11368
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Thanh
>
> List of UDT can't be updated properly when using USING TIMESTAMP
> Observe:
> {code}
> cqlsh:t360> CREATE TYPE fullname ( 
> ... fname text, 
> ... lname text 
> ... );
> cqlsh:t360> CREATE TABLE users ( 
> ... id text PRIMARY KEY, 
> ... names list>, 
> ... phone text 
> ... ); 
> cqlsh:t360> UPDATE users USING TIMESTAMP 1458019725701 SET names = [{ fname: 
> 'fname1', lname: 'lname1'},{ fname: 'fname2', lname: 'lname2'},{ fname: 
> 'fname3', lname: 'lname3'}] WHERE id='a'; 
> cqlsh:t360> select * from users;
> id | names | phone 
> +--+---
>  
> a | [{lname: 'lname1', fname: 'fname1'}, {lname: 'lname2', fname: 'fname2'}, 
> {lname: 'lname3', fname: 'fname3'}] | null
> (1 rows) 
> cqlsh:t360> UPDATE users USING TIMESTAMP 1458019725701 SET names = [{ fname: 
> 'fname1', lname: 'lname1'},{ fname: 'fname2', lname: 'lname2'},{ fname: 
> 'fname3', lname: 'lname3'}] WHERE id='a'; 
> cqlsh:t360> select * from users;
> id | names | phone 
> +--+---
>  
> a | [{lname: 'lname1', fname: 'fname1'}, {lname: 'lname2', fname: 'fname2'}, 
> {lname: 'lname3', fname: 'fname3'}, {lname: 'lname1', fname: 'fname1'}, 
> {lname: 'lname2', fname: 'fname2'}, {lname: 'lname3', fname: 'fname3'}] | null
> (1 rows)
> {code}
> => the list doesn't get replaced, it gets appended, which is not the 
> expected/desired result



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12058) dtest failure in cqlsh_tests.cqlsh_tests.TestCqlsh.test_describe

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12058:
-

 Summary: dtest failure in 
cqlsh_tests.cqlsh_tests.TestCqlsh.test_describe
 Key: CASSANDRA-12058
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12058
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1283/testReport/cqlsh_tests.cqlsh_tests/TestCqlsh/test_describe

Failed on CassCI build trunk_dtest #1283

Logs are attached.

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 726, 
in test_describe
self.execute(cql="DESCRIBE test.users2", expected_err="'users2' not found 
in keyspace 'test'")
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 986, 
in execute
self.check_response(err, expected_err)
  File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 999, 
in check_response
self.assertEqual(expected_lines, lines)
  File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 742, in assertListEqual
self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib/python2.7/unittest/case.py", line 724, in assertSequenceEqual
self.fail(msg)
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)
{code}

{code}
Error Message

Lists differ: ["'users2' not found in keyspa... != ["error: ('Unable to 
complete ...

First differing element 0:
'users2' not found in keyspace 'test'
error: ('Unable to complete the operation against any hosts', {})

- ["'users2' not found in keyspace 'test'"]
+ ["error: ('Unable to complete the operation against any hosts', {})"]
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344516#comment-15344516
 ] 

T Jake Luciani commented on CASSANDRA-9779:
---


But we can force the user to always send all the data and block any partial 
writes/updates. 
I guess my point was I think that's enough or the best we can do ATM.  The 
alternative being proposed, I think, was just state the contract.

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id

[Olivier Michallat (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344506#comment-15344506
 ] 

Olivier Michallat commented on CASSANDRA-10786:
---

>From a driver maintainer's point of view, I also agree that (1) sounds better.

I was wondering if we needed to version beta versions themselves, but I don't 
think that's necessary: "v5 beta" will mean different things at different 
times, and there might be errors when an older driver version (supporting "v5 
beta with new feature A") connects to a newer C* version (supporting "v5 beta 
with new features A and B"). But with the provision that "bad things can happen 
when you're in beta" that's acceptable, so we don't need to make things any 
more complicated.

I was also wondering about protocol negotiation, but there shouldn't be any 
issue provided that:

* if the client doesn't force a particular version, the driver should try with 
the latest *stable* version.
* if the server doesn't support the version requested by the driver, it should 
reply with the latest *stable* version.

> Include hash of result set metadata in prepared statement id
> 
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
>  Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a 
> prepared statement when the table is altered, to force clients to update 
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host. 
> The first client to execute the query after the cache was invalidated will 
> receive an UNPREPARED response, re-prepare, and update its local metadata. 
> But other clients might miss it entirely (the MD5 hasn't changed), and they 
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123, 
> clientA and clientB both have a cache of the metadata (columns b and c) 
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response, 
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been 
> populated again, the query succeeds. But clientB still has not updated its 
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set 
> metadata in the md5. This way the md5 would change at step 3, and any client 
> using the old md5 would get an UNPREPARED, regardless of whether another 
> client already reprepared.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12057) dtest failure in repair_tests.repair_test.TestRepair.repair_after_upgrade_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12057:
-

 Summary: dtest failure in 
repair_tests.repair_test.TestRepair.repair_after_upgrade_test
 Key: CASSANDRA-12057
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12057
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log

example failure:

http://cassci.datastax.com/job/trunk_dtest/1279/testReport/repair_tests.repair_test/TestRepair/repair_after_upgrade_test

Failed on CassCI build trunk_dtest #1279

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 217, 
in repair_after_upgrade_test
self._repair_and_verify(True)
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 287, 
in _repair_and_verify
self.check_rows_on_node(node3, 2000, missings=[1000])
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 70, 
in check_rows_on_node
self.assertEqual(len(result), rows)
  File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
raise self.failureException(msg)
'2001 != 2000
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344490#comment-15344490
 ] 

Sylvain Lebresne commented on CASSANDRA-9779:
-

bq. The big win is for MV

Well, that kinds of bring the point that "just" forcing the user to provide all 
columns actually doesn't work, because you still need to respect timestamps if 
there is multiple updates and so "keeping" the first hit, or in the case of 
MVs, you can't assume the update will have a higher timestamp than what's in 
store.

So you do need to make some assumption that there is no override, and we can't 
valid that, which I don't love.

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344473#comment-15344473
 ] 

T Jake Luciani commented on CASSANDRA-9779:
---

The *big win* is for MV and probably 2i since we don't need to read before 
write anymore.



> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344463#comment-15344463
 ] 

Sylvain Lebresne commented on CASSANDRA-9779:
-

Shouldn't our first step to validate that the optimization we have in mind 
actually make a meaningful difference (without having to bend a benchmark too 
hard to show benefits)? It seems clear to me that this will add complexity from 
the user point of view (it's a new concept that will either have good 
footshooting potential (if we were to just trust the user to insert only 
without checking it) and be annoying to use (if we force all columns every 
time)), so it sounds to me like we would need to demonstrate fairly big 
performance benefits to be worth doing (keep in mind that once we add such 
thing, we can't easily remove it, even if the improvement become obsolete).

tl;dr, I don't love that whole idea as I think it adds complexity from the user 
point of view (don't get me wrong, if we could validate this at insert time, 
I'd be a lot more fan, but we can't), and I'm wondering if given DTCS and other 
optimization we have internally this really bring that much to the table. 

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344435#comment-15344435
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

Yes, I'm currently running multiplexed tests. If they're clean, I'm submitting 
patch shortly.

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11911) CQLSSTableWriter should allow for unset fields

[Alex Petrov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344438#comment-15344438
 ] 

Alex Petrov commented on CASSANDRA-11911:
-

Thank you for the review! 

I've addressed the problems you've listed and re-ran the tests:

||[trunk|https://github.com/ifesdjeen/cassandra/tree/11911-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11911-trunk-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11911-trunk-dtest/]|

> CQLSSTableWriter should allow for unset fields
> --
>
> Key: CASSANDRA-11911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11911
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Cassandra 3.0.6
>Reporter: Matt Kopit
>Assignee: Alex Petrov
>  Labels: lhf
>
> If you are using CQLSSTableWriter to bulk load data into sstables the only 
> way to handle fields without values is by setting them to NULL, which results 
> in the generation of a tombstoned field in the resulting sstable. For a large 
> dataset this can result in a large number of tombstones.
> CQLSSTableWriter is currently instantiated with a single INSERT statement, so 
> it's not an option to modify the insert statement to specify different fields 
> on a per-row basis.
> Here are three potential solutions to this problem:
> 1. Change the default behavior of how NULLs are handled so those fields are 
> treated as UNSET and will never be written to the sstable.
> 2. Create a configuration option for CQLSSTableWriter that governs whether 
> NULLs should be ignored.
> 3. Invent a new constant that represents an UNSET value which can be used in 
> place of NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12056) dtest failure in pushed_notifications_test.TestPushedNotifications.add_and_remove_node_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12056:
-

 Summary: dtest failure in 
pushed_notifications_test.TestPushedNotifications.add_and_remove_node_test
 Key: CASSANDRA-12056
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12056
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node2.log

example failure:

http://cassci.datastax.com/job/cassandra-2.1_dtest/482/testReport/pushed_notifications_test/TestPushedNotifications/add_and_remove_node_test

Failed on CassCI build cassandra-2.1_dtest #482

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/pushed_notifications_test.py", line 
242, in add_and_remove_node_test
self.assertEquals(2, len(notifications), notifications)
  File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
raise self.failureException(msg)
"[{'change_type': u'NEW_NODE', 'address': ('127.0.0.2', 9042)}]
{code}

Logs are attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[Tupshin Harper (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344434#comment-15344434
 ] 

Tupshin Harper commented on CASSANDRA-9779:
---

Basically we are talking about frozen rows(as analogy to frozen collections), 
and I am very much in favor of this. *Many* use cases, particularly IoT, would 
be able to use such an optimization while still benefiting from representing 
data in highly structured columns.

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344430#comment-15344430
 ] 

T Jake Luciani commented on CASSANDRA-11937:


Makes sense. So clear the Builder when build() is called?

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9779) Append-only optimization

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344405#comment-15344405
 ] 

T Jake Luciani edited comment on CASSANDRA-9779 at 6/22/16 2:50 PM:


bq.  if you violate the INSERTS ONLY contract by updating existing rows, 
Cassandra will give you one of those versions back when you query it, but not 
necessarily the most recent.

It sounds like you are saying there are no guarantees.   

I've given this some thought and I think the best approach in which we can 
syntactically "do something" is to combine this ticket with the idea [~thobbs] 
touched on in CASSANDRA-9928. This might be what you are describing we should 
do but I'll just restate it.

bq. One possible solution is to require that all non-PK columns that are in a 
view PK be updated simultaneously. T Jake Luciani mentioned possible problems 
from read repair, but it seems like, with this restriction in place, any read 
repairs would end up repairing all non-PK columns at once.

Basically, this would add a mode where we only allow INSERT of *all* columns 
every time.  While this sounds restrictive, it also forces the user to deal 
with the fact that making updates conceptually/logistically hard since we would 
kick out all client mutations that don't specify all columns.  Sure you could 
subvert this but to me at least, the server can alert the user that updating 
existing data, as they can with other tables, is hard.

So the proposal is:

  * Add a table level flag/syntax to mark that a table is INSERT ONLY (which 
can be altered if there's an emergency).
  * Reject any INSERTS/UPSERTS that do not specify all columns
  * Possibly always return the earliest row if there is a conflict.
  * When writing to the memtable we can add a putIfAbsent method to 
reject/ignore updates (to cover some minimal bases) 


was (Author: tjake):
bq.  if you violate the INSERTS ONLY contract by updating existing rows, 
Cassandra will give you one of those versions back when you query it, but not 
necessarily the most recent.

It sounds like you are saying there are no guarantees.   

I've given this some thought and I think the best approach in which we can 
syntactically "do something" is to combine this ticket with the idea [~thobbs] 
touched on in CASSANDRA-9928. This might be what you are describing we should 
do but I'll just restate it.

bq. One possible solution is to require that all non-PK columns that are in a 
view PK be updated simultaneously. T Jake Luciani mentioned possible problems 
from read repair, but it seems like, with this restriction in place, any read 
repairs would end up repairing all non-PK columns at once.

Basically, this would add a mode where we INSERT *all* columns every time.  
While this sounds restrictive, it also forces the user to deal with the fact 
that making updates conceptually/logistically hard since we would kick out all 
client mutations that don't specify all columns.  Sure you could subvert this 
but to me at least, the server can alert the user that updating existing data, 
as they can with other tables, is hard.

So the proposal is:

  * Add a table level flag/syntax to mark that a table is INSERT ONLY (which 
can be altered if there's an emergency).
  * Reject any INSERTS/UPSERTS that do not specify all columns
  * Possibly always return the earliest row if there is a conflict.
  * When writing to the memtable we can add a putIfAbsent method to 
reject/ignore updates (to cover some minimal bases) 

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9779) Append-only optimization

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344405#comment-15344405
 ] 

T Jake Luciani edited comment on CASSANDRA-9779 at 6/22/16 2:49 PM:


bq.  if you violate the INSERTS ONLY contract by updating existing rows, 
Cassandra will give you one of those versions back when you query it, but not 
necessarily the most recent.

It sounds like you are saying there are no guarantees.   

I've given this some thought and I think the best approach in which we can 
syntactically "do something" is to combine this ticket with the idea [~thobbs] 
touched on in CASSANDRA-9928. This might be what you are describing we should 
do but I'll just restate it.

bq. One possible solution is to require that all non-PK columns that are in a 
view PK be updated simultaneously. T Jake Luciani mentioned possible problems 
from read repair, but it seems like, with this restriction in place, any read 
repairs would end up repairing all non-PK columns at once.

Basically, this would add a mode where we INSERT *all* columns every time.  
While this sounds restrictive, it also forces the user to deal with the fact 
that making updates conceptually/logistically hard since we would kick out all 
client mutations that don't specify all columns.  Sure you could subvert this 
but to me at least, the server can alert the user that updating existing data, 
as they can with other tables, is hard.

So the proposal is:

  * Add a table level flag/syntax to mark that a table is INSERT ONLY (which 
can be altered if there's an emergency).
  * Reject any INSERTS/UPSERTS that do not specify all columns
  * Possibly always return the earliest row if there is a conflict.
  * When writing to the memtable we can add a putIfAbsent method to 
reject/ignore updates (to cover some minimal bases) 


was (Author: tjake):
bq.  if you violate the INSERTS ONLY contract by updating existing rows, 
Cassandra will give you one of those versions back when you query it, but not 
necessarily the most recent.

It sounds like you are saying there are no guarantees.   

I've given this some thought and I think the best approach in which we can 
syntactically "do something" is to combine this ticket with the idea [~thobbs] 
touched on in CASSANDRA-9928. This might be what you are describing we should 
do but I'll just restate it.

bq. One possible solution is to require that all non-PK columns that are in a 
view PK be updated simultaneously. T Jake Luciani mentioned possible problems 
from read repair, but it seems like, with this restriction in place, any read 
repairs would end up repairing all non-PK columns at once.

Basically, this would add a mode where we INSERT *all* columns every time.  
While this sounds restrictive, it also forces the user to deal with the fact 
that making updates conceptually/logistically hard since we would kick out all 
client mutations that don't specify all columns.  Sure you could subvert this 
but to me at least, the server can alert the user that updating existing data 
as in other tables is hard.

So the proposal is:

  * Add a table level flag/syntax to mark that a table is INSERT ONLY (which 
can be altered if there's an emergency).
  * Reject any INSERTS/UPSERTS that do not specify all columns
  * Possibly always return the earliest row if there is a conflict.
  * When writing to the memtable we can add a putIfAbsent method to 
reject/ignore updates (to cover some minimal bases) 

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9779) Append-only optimization

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344405#comment-15344405
 ] 

T Jake Luciani edited comment on CASSANDRA-9779 at 6/22/16 2:46 PM:


bq.  if you violate the INSERTS ONLY contract by updating existing rows, 
Cassandra will give you one of those versions back when you query it, but not 
necessarily the most recent.

It sounds like you are saying there are no guarantees.   

I've given this some thought and I think the best approach in which we can 
syntactically "do something" is to combine this ticket with the idea [~thobbs] 
touched on in CASSANDRA-9928. This might be what you are describing we should 
do but I'll just restate it.

bq. One possible solution is to require that all non-PK columns that are in a 
view PK be updated simultaneously. T Jake Luciani mentioned possible problems 
from read repair, but it seems like, with this restriction in place, any read 
repairs would end up repairing all non-PK columns at once.

Basically, this would add a mode where we INSERT *all* columns every time.  
While this sounds restrictive, it also forces the user to deal with the fact 
that making updates conceptually/logistically hard since we would kick out all 
client mutations that don't specify all columns.  Sure you could subvert this 
but to me at least, the server can alert the user that updating existing data 
as in other tables is hard.

So the proposal is:

  * Add a table level flag/syntax to mark that a table is INSERT ONLY (which 
can be altered if there's an emergency).
  * Reject any INSERTS/UPSERTS that do not specify all columns
  * Possibly always return the earliest row if there is a conflict.
  * When writing to the memtable we can add a putIfAbsent method to 
reject/ignore updates (to cover some minimal bases) 


was (Author: tjake):
.bq  if you violate the INSERTS ONLY contract by updating existing rows, 
Cassandra will give you one of those versions back when you query it, but not 
necessarily the most recent.

It sounds like you are saying there are no guarantees.   

I've given this some thought and I think the best approach in which we can 
syntactically "do something" is to combine this ticket with the idea [~thobbs] 
touched on in CASSANDRA-9928. This might be what you are describing we should 
do but I'll just restate it.

bq. One possible solution is to require that all non-PK columns that are in a 
view PK be updated simultaneously. T Jake Luciani mentioned possible problems 
from read repair, but it seems like, with this restriction in place, any read 
repairs would end up repairing all non-PK columns at once.

Basically, this would add a mode where we INSERT *all* columns every time.  
While this sounds restrictive, it also forces the user to deal with the fact 
that making updates conceptually/logistically hard since we would kick out all 
client mutations that don't specify all columns.  Sure you could subvert this 
but to me at least, the server can alert the user that updating existing data 
as in other tables is hard.

So the proposal is:

  * Add a table level flag/syntax to mark that a table is INSERT ONLY (which 
can be altered if there's an emergency).
  * Reject any INSERTS/UPSERTS that do not specify all columns
  * Possibly always return the earliest row if there is a conflict.
  * When writing to the memtable we can add a putIfAbsent method to 
reject/ignore updates (to cover some minimal bases) 

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344405#comment-15344405
 ] 

T Jake Luciani commented on CASSANDRA-9779:
---

.bq  if you violate the INSERTS ONLY contract by updating existing rows, 
Cassandra will give you one of those versions back when you query it, but not 
necessarily the most recent.

It sounds like you are saying there are no guarantees.   

I've given this some thought and I think the best approach in which we can 
syntactically "do something" is to combine this ticket with the idea [~thobbs] 
touched on in CASSANDRA-9928. This might be what you are describing we should 
do but I'll just restate it.

bq. One possible solution is to require that all non-PK columns that are in a 
view PK be updated simultaneously. T Jake Luciani mentioned possible problems 
from read repair, but it seems like, with this restriction in place, any read 
repairs would end up repairing all non-PK columns at once.

Basically, this would add a mode where we INSERT *all* columns every time.  
While this sounds restrictive, it also forces the user to deal with the fact 
that making updates conceptually/logistically hard since we would kick out all 
client mutations that don't specify all columns.  Sure you could subvert this 
but to me at least, the server can alert the user that updating existing data 
as in other tables is hard.

So the proposal is:

  * Add a table level flag/syntax to mark that a table is INSERT ONLY (which 
can be altered if there's an emergency).
  * Reject any INSERTS/UPSERTS that do not specify all columns
  * Possibly always return the earliest row if there is a conflict.
  * When writing to the memtable we can add a putIfAbsent method to 
reject/ignore updates (to cover some minimal bases) 

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11845) Hanging repair in cassandra 2.2.4

[vin01 (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

vin01 updated CASSANDRA-11845:
--
Description: 
 So after increasing the streaming_timeout_in_ms value to 3 hours, i was able 
to avoid the socketTimeout errors i was getting earlier 
(https://issues.apAache.org/jira/browse/CASSANDRA-11826), but now the issue is 
repair just stays stuck.

current status :-

[2016-05-19 05:52:50,835] Repair session a0e590e1-1d99-11e6-9d63-b717b380ffdd 
for range (-3309358208555432808,-3279958773585646585] finished (progress: 54%)
[2016-05-19 05:53:09,446] Repair session a0e590e3-1d99-11e6-9d63-b717b380ffdd 
for range (8149151263857514385,8181801084802729407] finished (progress: 55%)
[2016-05-19 05:53:13,808] Repair session a0e5b7f1-1d99-11e6-9d63-b717b380ffdd 
for range (3372779397996730299,3381236471688156773] finished (progress: 55%)
[2016-05-19 05:53:27,543] Repair session a0e5b7f3-1d99-11e6-9d63-b717b380ffdd 
for range (-4182952858113330342,-4157904914928848809] finished (progress: 55%)
[2016-05-19 05:53:41,128] Repair session a0e5df00-1d99-11e6-9d63-b717b380ffdd 
for range (6499366179019889198,6523760493740195344] finished (progress: 55%)


And its 10:46:25 Now, almost 5 hours since it has been stuck right there.

Earlier i could see repair session going on in system.log but there are no logs 
coming in right now, all i get in logs is regular index summary redistribution 
logs.


Last logs for repair i saw in logs :-

INFO  [RepairJobTask:5] 2016-05-19 05:53:41,125 RepairJob.java:152 - [repair 
#a0e5df00-1d99-11e6-9d63-b717b380ffdd] TABLE_NAME is fully synced
INFO  [RepairJobTask:5] 2016-05-19 05:53:41,126 RepairSession.java:279 - 
[repair #a0e5df00-1d99-11e6-9d63-b717b380ffdd] Session completed successfully
INFO  [RepairJobTask:5] 2016-05-19 05:53:41,126 RepairRunnable.java:232 - 
Repair session a0e5df00-1d99-11e6-9d63-b717b380ffdd for range 
(6499366179019889198,6523760493740195344] finished

Its an incremental repair, and in "nodetool netstats" output i can see logs 
like :-



Repair e3055fb0-1d9d-11e6-9d63-b717b380ffdd
/Node-2
Receiving 8 files, 1093461 bytes total. Already received 8 files, 
1093461 bytes total

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80872-big-Data.db
 399475/399475 bytes(100%) received from idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80879-big-Data.db
 53809/53809 bytes(100%) received from idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80878-big-Data.db
 89955/89955 bytes(100%) received from idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80881-big-Data.db
 168790/168790 bytes(100%) received from idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80886-big-Data.db
 107785/107785 bytes(100%) received from idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80880-big-Data.db
 52889/52889 bytes(100%) received from idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80884-big-Data.db
 148882/148882 bytes(100%) received from idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80883-big-Data.db
 71876/71876 bytes(100%) received from idx:0/Node-2
Sending 5 files, 863321 bytes total. Already sent 5 files, 863321 bytes 
total

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/la-73168-big-Data.db
 161895/161895 bytes(100%) sent to idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/la-72604-big-Data.db
 399865/399865 bytes(100%) sent to idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/la-73147-big-Data.db
 149066/149066 bytes(100%) sent to idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/la-72682-big-Data.db
 126000/126000 bytes(100%) sent to idx:0/Node-2

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/la-73173-big-Data.db
 26495/26495 bytes(100%) sent to idx:0/Node-2
Repair c0c8af20-1d9c-11e6-9d63-b717b380ffdd
/Node-3
Receiving 11 files, 13896288 bytes total. Already received 11 files, 
13896288 bytes total

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-79186-big-Data.db
 1598874/1598874 bytes(100%) received from idx:0/Node-3

/data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-79196-big-Data.db
 736365/736365 bytes(100%) received 

[jira] [Commented] (CASSANDRA-11845) Hanging repair in cassandra 2.2.4

[vin01 (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344344#comment-15344344
 ] 

vin01 commented on CASSANDRA-11845:
---

It worked !!

So tuning of following three things made it work :-

1. increasing streaming_timeout_in_ms value to 6 hours.

2. sudo sysctl -w net.ipv4.tcp_keepalive_time=60 
net.ipv4.tcp_keepalive_probes=3 net.ipv4.tcp_keepalive_intvl=10
(as per 
https://docs.datastax.com/en/cassandra/2.2/cassandra/troubleshooting/trblshootIdleFirewall.html)

3. Increasing value of vm.max_map_count 
(https://docs.datastax.com/en/cassandra/2.2/cassandra/install/installRecommendSettings.html.)

Thanks again Paulo!

> Hanging repair in cassandra 2.2.4
> -
>
> Key: CASSANDRA-11845
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11845
> Project: Cassandra
>  Issue Type: Bug
>  Components: Streaming and Messaging
> Environment: Centos 6
>Reporter: vin01
>Priority: Minor
> Attachments: cassandra-2.2.4.error.log, system.log
>
>
> So after increasing the streaming_timeout_in_ms value to 3 hours, i was able 
> to avoid the socketTimeout errors i was getting earlier 
> (https://issues.apAache.org/jira/browse/CASSANDRA-11826), but now the issue 
> is repair just stays stuck.
> current status :-
> [2016-05-19 05:52:50,835] Repair session a0e590e1-1d99-11e6-9d63-b717b380ffdd 
> for range (-3309358208555432808,-3279958773585646585] finished (progress: 54%)
> [2016-05-19 05:53:09,446] Repair session a0e590e3-1d99-11e6-9d63-b717b380ffdd 
> for range (8149151263857514385,8181801084802729407] finished (progress: 55%)
> [2016-05-19 05:53:13,808] Repair session a0e5b7f1-1d99-11e6-9d63-b717b380ffdd 
> for range (3372779397996730299,3381236471688156773] finished (progress: 55%)
> [2016-05-19 05:53:27,543] Repair session a0e5b7f3-1d99-11e6-9d63-b717b380ffdd 
> for range (-4182952858113330342,-4157904914928848809] finished (progress: 55%)
> [2016-05-19 05:53:41,128] Repair session a0e5df00-1d99-11e6-9d63-b717b380ffdd 
> for range (6499366179019889198,6523760493740195344] finished (progress: 55%)
> And its 10:46:25 Now, almost 5 hours since it has been stuck right there.
> Earlier i could see repair session going on in system.log but there are no 
> logs coming in right now, all i get in logs is regular index summary 
> redistribution logs.
> Last logs for repair i saw in logs :-
> INFO  [RepairJobTask:5] 2016-05-19 05:53:41,125 RepairJob.java:152 - [repair 
> #a0e5df00-1d99-11e6-9d63-b717b380ffdd] TABLE_NAME is fully synced
> INFO  [RepairJobTask:5] 2016-05-19 05:53:41,126 RepairSession.java:279 - 
> [repair #a0e5df00-1d99-11e6-9d63-b717b380ffdd] Session completed successfully
> INFO  [RepairJobTask:5] 2016-05-19 05:53:41,126 RepairRunnable.java:232 - 
> Repair session a0e5df00-1d99-11e6-9d63-b717b380ffdd for range 
> (6499366179019889198,6523760493740195344] finished
> Its an incremental repair, and in "nodetool netstats" output i can see logs 
> like :-
> Repair e3055fb0-1d9d-11e6-9d63-b717b380ffdd
> /Node-2
> Receiving 8 files, 1093461 bytes total. Already received 8 files, 
> 1093461 bytes total
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80872-big-Data.db
>  399475/399475 bytes(100%) received from idx:0/Node-2
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80879-big-Data.db
>  53809/53809 bytes(100%) received from idx:0/Node-2
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80878-big-Data.db
>  89955/89955 bytes(100%) received from idx:0/Node-2
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80881-big-Data.db
>  168790/168790 bytes(100%) received from idx:0/Node-2
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80886-big-Data.db
>  107785/107785 bytes(100%) received from idx:0/Node-2
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80880-big-Data.db
>  52889/52889 bytes(100%) received from idx:0/Node-2
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80884-big-Data.db
>  148882/148882 bytes(100%) received from idx:0/Node-2
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/tmp-la-80883-big-Data.db
>  71876/71876 bytes(100%) received from idx:0/Node-2
> Sending 5 files, 863321 bytes total. Already sent 5 files, 863321 
> bytes total
> 
> /data/cassandra/data/KEYSPACE_NAME/TABLE_NAME-01ad9750723e11e4bfe0d3887930a87c/la-73168-big-Data.db
>  161895/161895 bytes(100%) sent to idx:0/Node-2
> 
> 

[jira] [Commented] (CASSANDRA-9779) Append-only optimization

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344339#comment-15344339
 ] 

Jonathan Ellis commented on CASSANDRA-9779:
---

Easy enough to ignore merges of the same value.  Same problem + solution in the 
case of timed-out updates that get retried.

> Append-only optimization
> 
>
> Key: CASSANDRA-9779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9779
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
> Fix For: 3.x
>
>
> Many common workloads are append-only: that is, they insert new rows but do 
> not update existing ones.  However, Cassandra has no way to infer this and so 
> it must treat all tables as if they may experience updates in the future.
> If we added syntax to tell Cassandra about this ({{WITH INSERTS ONLY}} for 
> instance) then we could do a number of optimizations:
> - Compaction would only need to worry about defragmenting partitions, not 
> rows.  We could default to DTCS or similar.
> - CollationController could stop scanning sstables as soon as it finds a 
> matching row
> - Most importantly, materialized views wouldn't need to worry about deleting 
> prior values, which would eliminate the majority of the MV overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)