[01/10] cassandra git commit: BATCH statement is broken in cqlsh

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 488db6f50 -> ae5108645
  refs/heads/cassandra-2.2 53d044917 -> 979179609
  refs/heads/cassandra-3.0 00264e854 -> 29c6f1d96
  refs/heads/trunk 1e66e97ca -> e234b314a


BATCH statement is broken in cqlsh

patch by Stefania; reviewed by Robert Stupp for CASSANDRA-10272


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae510864
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae510864
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae510864

Branch: refs/heads/cassandra-2.1
Commit: ae5108645fc5e04888ab5cf08d3a03730b90ff30
Parents: 488db6f
Author: Stefania 
Authored: Sun Sep 13 11:33:14 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:33:14 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4e0df42..dff47fc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.10
+ * BATCH statement is broken in cqlsh (CASSANDRA-10272)
  * Added configurable warning threshold for GC duration (CASSANDRA-8907)
  * (cqlsh) Make cqlsh PEP8 compliant (CASSANDRA-10066)
  * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 74b1281..bc4f2ef 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -1221,7 +1221,7 @@ def username_name_completer(ctxt, cass):
 return "'%s'" % name
 
 # disable completion for CREATE USER.
-if ctxt.matched[0][0] == 'create':
+if ctxt.matched[0][1].upper() == 'CREATE':
 return [Hint('')]
 
 session = cass.session

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cqlhandling.py
--
diff --git a/pylib/cqlshlib/cqlhandling.py b/pylib/cqlshlib/cqlhandling.py
index 31c636c..5fe311f 100644
--- a/pylib/cqlshlib/cqlhandling.py
+++ b/pylib/cqlshlib/cqlhandling.py
@@ -138,9 +138,9 @@ class CqlParsingRuleSet(pylexotron.ParsingRuleSet):
 else:
 output.append(stmt)
 if len(stmt) > 2:
-if stmt[-3][0] == 'apply':
+if stmt[-3][1].upper() == 'APPLY':
 in_batch = False
-elif stmt[0][0] == 'begin':
+elif stmt[0][1].upper() == 'BEGIN':
 in_batch = True
 return output, in_batch
 

[05/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[snazy]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/97917960
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/97917960
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/97917960

Branch: refs/heads/cassandra-3.0
Commit: 9791796096cc2f61444169721bbc0ca03d9509e5
Parents: 53d0449 ae51086
Author: Robert Stupp 
Authored: Sun Sep 13 11:34:31 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:34:31 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/CHANGES.txt
--
diff --cc CHANGES.txt
index ffae4d9,dff47fc..bd24781
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,7 +1,17 @@@
 -2.1.10
 +2.2.2
 + * Defer default role manager setup until all nodes are on 2.2+ 
(CASSANDRA-9761)
 + * Cancel transaction for sstables we wont redistribute index summary
 +   for (CASSANDRA-10270)
 + * Handle missing RoleManager in config after upgrade to 2.2 
(CASSANDRA-10209) 
 + * Retry snapshot deletion after compaction and gc on Windows 
(CASSANDRA-10222)
 + * Fix failure to start with space in directory path on Windows 
(CASSANDRA-10239)
 + * Fix repair hang when snapshot failed (CASSANDRA-10057)
 + * Fall back to 1/4 commitlog volume for commitlog_total_space on small disks
 +   (CASSANDRA-10199)
 +Merged from 2.1:
+  * BATCH statement is broken in cqlsh (CASSANDRA-10272)
   * Added configurable warning threshold for GC duration (CASSANDRA-8907)
 - * (cqlsh) Make cqlsh PEP8 compliant (CASSANDRA-10066)
 + * (cqlsh) Make cqlsh PEP8 Compliant (CASSANDRA-10066)
   * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)
   * Scrub, Cleanup and Upgrade do not unmark compacting until all operations
 have completed, regardless of the occurence of exceptions (CASSANDRA-10274)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/pylib/cqlshlib/cql3handling.py
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/pylib/cqlshlib/cqlhandling.py
--

[09/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[snazy]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29c6f1d9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29c6f1d9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29c6f1d9

Branch: refs/heads/trunk
Commit: 29c6f1d96f07faf5788779416b56eb8b041035b4
Parents: 00264e8 9791796
Author: Robert Stupp 
Authored: Sun Sep 13 11:35:23 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:35:58 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/29c6f1d9/CHANGES.txt
--
diff --cc CHANGES.txt
index d367b85,bd24781..bacedaf
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,42 -1,8 +1,43 @@@
 -2.2.2
 +3.0.0-rc1
 + * Add "shadowable" row tombstones to deal with mv timestamp issues 
(CASSANDRA-10261)
 + * CFS.loadNewSSTables() broken for pre-3.0 sstables
 + * Cache selected index in read command to reduce lookups (CASSANDRA-10215)
 + * Small optimizations of sstable index serialization (CASSANDRA-10232)
 + * Support for both encrypted and unencrypted native transport connections 
(CASSANDRA-9590)
 +Merged from 2.2:
   * Defer default role manager setup until all nodes are on 2.2+ 
(CASSANDRA-9761)
 + * Handle missing RoleManager in config after upgrade to 2.2 (CASSANDRA-10209)
 +Merged from 2.1:
++ * BATCH statement is broken in cqlsh (CASSANDRA-10272)
 + * (cqlsh) Make cqlsh PEP8 Compliant (CASSANDRA-10066)
 + * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)
 + * Scrub, Cleanup and Upgrade do not unmark compacting until all operations
 +   have completed, regardless of the occurence of exceptions (CASSANDRA-10274)
 +
 +
 +3.0.0-beta2
 + * Fix columns returned by AbstractBtreePartitions (CASSANDRA-10220)
 + * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
 + * Fix startup error when upgrading nodes (CASSANDRA-10136)
 + * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)
 + * Improve batchlog write patch (CASSANDRA-9673)
 + * Re-apply MaterializedView updates on commitlog replay (CASSANDRA-10164)
 + * Require AbstractType.isByteOrderComparable declaration in constructor 
(CASSANDRA-9901)
 + * Avoid digest mismatch on upgrade to 3.0 (CASSANDRA-9554)
 + * Fix Materialized View builder when adding multiple MVs (CASSANDRA-10156)
 + * Choose better poolingOptions for protocol v4 in cassandra-stress 
(CASSANDRA-10182)
 + * Fix LWW bug affecting Materialized Views (CASSANDRA-10197)
 + * Ensures frozen sets and maps are always sorted (CASSANDRA-10162)
 + * Don't deadlock when flushing CFS backed custom indexes (CASSANDRA-10181)
 + * Fix double flushing of secondary index tables (CASSANDRA-10180)
 + * Fix incorrect handling of range tombstones in thrift (CASSANDRA-10046)
 + * Only use batchlog when paired materialized view replica is remote 
(CASSANDRA-10061)
 + * Reuse TemporalRow when updating multiple MaterializedViews 
(CASSANDRA-10060)
 + * Validate gc_grace_seconds for batchlog writes and MVs (CASSANDRA-9917)
 + * Fix sstablerepairedset (CASSANDRA-10132)
 +Merged from 2.2:
   * Cancel transaction for sstables we wont redistribute index summary
 for (CASSANDRA-10270)
 - * Handle missing RoleManager in config after upgrade to 2.2 
(CASSANDRA-10209) 
   * Retry snapshot deletion after compaction and gc on Windows 
(CASSANDRA-10222)
   * Fix failure to start with space in directory path on Windows 
(CASSANDRA-10239)
   * Fix repair hang when snapshot failed (CASSANDRA-10057)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/29c6f1d9/pylib/cqlshlib/cql3handling.py
--

[jira] [Commented] (CASSANDRA-10272) BATCH statement is broken in cqlsh

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742422#comment-14742422
 ] 

Robert Stupp commented on CASSANDRA-10272:
--

+1

committed as ae5108645fc5e04888ab5cf08d3a03730b90ff30

> BATCH statement is broken in cqlsh
> --
>
> Key: CASSANDRA-10272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10272
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: CentOS 7.1/x64
>Reporter: Vovodroid
>Assignee: Stefania
> Fix For: 3.0.0 rc1, 2.1.10, 2.2.2
>
> Attachments: 10272-2.1.txt
>
>
> BEGIN BATCH  APPLY BATCH is not parsed correctly.
> Steps:
> {code}
> CREATE KEYSPACE Excelsior  WITH 
> REPLICATION={'class':'SimpleStrategy','replication_factor':1};
> CREATE TABLE excelsior.data (id int primary key);
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0); APPLY BATCH ;
> {code}
> Error
> {code}
> SyntaxException:  message="line 0:-1 mismatched input '' expecting K_APPLY">
> SyntaxException:  message="line 1:0 no viable alternative at input 'APPLY' ([APPLY]...)">
> {code}
> While 
> {code}
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0)  APPLY BATCH ;
> {code}
> without *;* after insert works.
> Consequently neither
> {code}
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0);INSERT INTO 
> excelsior.data (id) VALUES (0); APPLY BATCH ;
> {code}
> Error:
> {code}
> SyntaxException:  message="line 0:-1 mismatched input '' expecting K_APPLY">
> SyntaxException:  message="line 1:0 no viable alternative at input 'APPLY' ([APPLY]...)">
> {code}
> nor
> {code}
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0);INSERT INTO 
> excelsior.data (id) VALUES (0) APPLY BATCH ;
> {code}
> Error
> {code}
> SyntaxException:  message="line 0:-1 mismatched input '' expecting K_APPLY">
> SyntaxException:  message="line 1:43 missing EOF at 'APPLY' (...(id) VALUES (0) [APPLY] 
> BATCH...)">
> {code}
> works.
> It was OK in 2.2.0 and 3.0 beta 1.
> 3.0-beta2-tentative also affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[06/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[snazy]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/97917960
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/97917960
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/97917960

Branch: refs/heads/cassandra-2.2
Commit: 9791796096cc2f61444169721bbc0ca03d9509e5
Parents: 53d0449 ae51086
Author: Robert Stupp 
Authored: Sun Sep 13 11:34:31 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:34:31 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/CHANGES.txt
--
diff --cc CHANGES.txt
index ffae4d9,dff47fc..bd24781
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,7 +1,17 @@@
 -2.1.10
 +2.2.2
 + * Defer default role manager setup until all nodes are on 2.2+ 
(CASSANDRA-9761)
 + * Cancel transaction for sstables we wont redistribute index summary
 +   for (CASSANDRA-10270)
 + * Handle missing RoleManager in config after upgrade to 2.2 
(CASSANDRA-10209) 
 + * Retry snapshot deletion after compaction and gc on Windows 
(CASSANDRA-10222)
 + * Fix failure to start with space in directory path on Windows 
(CASSANDRA-10239)
 + * Fix repair hang when snapshot failed (CASSANDRA-10057)
 + * Fall back to 1/4 commitlog volume for commitlog_total_space on small disks
 +   (CASSANDRA-10199)
 +Merged from 2.1:
+  * BATCH statement is broken in cqlsh (CASSANDRA-10272)
   * Added configurable warning threshold for GC duration (CASSANDRA-8907)
 - * (cqlsh) Make cqlsh PEP8 compliant (CASSANDRA-10066)
 + * (cqlsh) Make cqlsh PEP8 Compliant (CASSANDRA-10066)
   * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)
   * Scrub, Cleanup and Upgrade do not unmark compacting until all operations
 have completed, regardless of the occurence of exceptions (CASSANDRA-10274)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/pylib/cqlshlib/cql3handling.py
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/pylib/cqlshlib/cqlhandling.py
--

[03/10] cassandra git commit: BATCH statement is broken in cqlsh

[snazy]

BATCH statement is broken in cqlsh

patch by Stefania; reviewed by Robert Stupp for CASSANDRA-10272


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae510864
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae510864
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae510864

Branch: refs/heads/cassandra-3.0
Commit: ae5108645fc5e04888ab5cf08d3a03730b90ff30
Parents: 488db6f
Author: Stefania 
Authored: Sun Sep 13 11:33:14 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:33:14 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4e0df42..dff47fc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.10
+ * BATCH statement is broken in cqlsh (CASSANDRA-10272)
  * Added configurable warning threshold for GC duration (CASSANDRA-8907)
  * (cqlsh) Make cqlsh PEP8 compliant (CASSANDRA-10066)
  * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 74b1281..bc4f2ef 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -1221,7 +1221,7 @@ def username_name_completer(ctxt, cass):
 return "'%s'" % name
 
 # disable completion for CREATE USER.
-if ctxt.matched[0][0] == 'create':
+if ctxt.matched[0][1].upper() == 'CREATE':
 return [Hint('')]
 
 session = cass.session

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cqlhandling.py
--
diff --git a/pylib/cqlshlib/cqlhandling.py b/pylib/cqlshlib/cqlhandling.py
index 31c636c..5fe311f 100644
--- a/pylib/cqlshlib/cqlhandling.py
+++ b/pylib/cqlshlib/cqlhandling.py
@@ -138,9 +138,9 @@ class CqlParsingRuleSet(pylexotron.ParsingRuleSet):
 else:
 output.append(stmt)
 if len(stmt) > 2:
-if stmt[-3][0] == 'apply':
+if stmt[-3][1].upper() == 'APPLY':
 in_batch = False
-elif stmt[0][0] == 'begin':
+elif stmt[0][1].upper() == 'BEGIN':
 in_batch = True
 return output, in_batch
 

[04/10] cassandra git commit: BATCH statement is broken in cqlsh

[snazy]

BATCH statement is broken in cqlsh

patch by Stefania; reviewed by Robert Stupp for CASSANDRA-10272


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae510864
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae510864
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae510864

Branch: refs/heads/trunk
Commit: ae5108645fc5e04888ab5cf08d3a03730b90ff30
Parents: 488db6f
Author: Stefania 
Authored: Sun Sep 13 11:33:14 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:33:14 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4e0df42..dff47fc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.10
+ * BATCH statement is broken in cqlsh (CASSANDRA-10272)
  * Added configurable warning threshold for GC duration (CASSANDRA-8907)
  * (cqlsh) Make cqlsh PEP8 compliant (CASSANDRA-10066)
  * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 74b1281..bc4f2ef 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -1221,7 +1221,7 @@ def username_name_completer(ctxt, cass):
 return "'%s'" % name
 
 # disable completion for CREATE USER.
-if ctxt.matched[0][0] == 'create':
+if ctxt.matched[0][1].upper() == 'CREATE':
 return [Hint('')]
 
 session = cass.session

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cqlhandling.py
--
diff --git a/pylib/cqlshlib/cqlhandling.py b/pylib/cqlshlib/cqlhandling.py
index 31c636c..5fe311f 100644
--- a/pylib/cqlshlib/cqlhandling.py
+++ b/pylib/cqlshlib/cqlhandling.py
@@ -138,9 +138,9 @@ class CqlParsingRuleSet(pylexotron.ParsingRuleSet):
 else:
 output.append(stmt)
 if len(stmt) > 2:
-if stmt[-3][0] == 'apply':
+if stmt[-3][1].upper() == 'APPLY':
 in_batch = False
-elif stmt[0][0] == 'begin':
+elif stmt[0][1].upper() == 'BEGIN':
 in_batch = True
 return output, in_batch
 

[07/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[snazy]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/97917960
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/97917960
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/97917960

Branch: refs/heads/trunk
Commit: 9791796096cc2f61444169721bbc0ca03d9509e5
Parents: 53d0449 ae51086
Author: Robert Stupp 
Authored: Sun Sep 13 11:34:31 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:34:31 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/CHANGES.txt
--
diff --cc CHANGES.txt
index ffae4d9,dff47fc..bd24781
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,7 +1,17 @@@
 -2.1.10
 +2.2.2
 + * Defer default role manager setup until all nodes are on 2.2+ 
(CASSANDRA-9761)
 + * Cancel transaction for sstables we wont redistribute index summary
 +   for (CASSANDRA-10270)
 + * Handle missing RoleManager in config after upgrade to 2.2 
(CASSANDRA-10209) 
 + * Retry snapshot deletion after compaction and gc on Windows 
(CASSANDRA-10222)
 + * Fix failure to start with space in directory path on Windows 
(CASSANDRA-10239)
 + * Fix repair hang when snapshot failed (CASSANDRA-10057)
 + * Fall back to 1/4 commitlog volume for commitlog_total_space on small disks
 +   (CASSANDRA-10199)
 +Merged from 2.1:
+  * BATCH statement is broken in cqlsh (CASSANDRA-10272)
   * Added configurable warning threshold for GC duration (CASSANDRA-8907)
 - * (cqlsh) Make cqlsh PEP8 compliant (CASSANDRA-10066)
 + * (cqlsh) Make cqlsh PEP8 Compliant (CASSANDRA-10066)
   * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)
   * Scrub, Cleanup and Upgrade do not unmark compacting until all operations
 have completed, regardless of the occurence of exceptions (CASSANDRA-10274)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/pylib/cqlshlib/cql3handling.py
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/97917960/pylib/cqlshlib/cqlhandling.py
--

[02/10] cassandra git commit: BATCH statement is broken in cqlsh

[snazy]

BATCH statement is broken in cqlsh

patch by Stefania; reviewed by Robert Stupp for CASSANDRA-10272


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae510864
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae510864
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae510864

Branch: refs/heads/cassandra-2.2
Commit: ae5108645fc5e04888ab5cf08d3a03730b90ff30
Parents: 488db6f
Author: Stefania 
Authored: Sun Sep 13 11:33:14 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:33:14 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4e0df42..dff47fc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.10
+ * BATCH statement is broken in cqlsh (CASSANDRA-10272)
  * Added configurable warning threshold for GC duration (CASSANDRA-8907)
  * (cqlsh) Make cqlsh PEP8 compliant (CASSANDRA-10066)
  * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 74b1281..bc4f2ef 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -1221,7 +1221,7 @@ def username_name_completer(ctxt, cass):
 return "'%s'" % name
 
 # disable completion for CREATE USER.
-if ctxt.matched[0][0] == 'create':
+if ctxt.matched[0][1].upper() == 'CREATE':
 return [Hint('')]
 
 session = cass.session

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae510864/pylib/cqlshlib/cqlhandling.py
--
diff --git a/pylib/cqlshlib/cqlhandling.py b/pylib/cqlshlib/cqlhandling.py
index 31c636c..5fe311f 100644
--- a/pylib/cqlshlib/cqlhandling.py
+++ b/pylib/cqlshlib/cqlhandling.py
@@ -138,9 +138,9 @@ class CqlParsingRuleSet(pylexotron.ParsingRuleSet):
 else:
 output.append(stmt)
 if len(stmt) > 2:
-if stmt[-3][0] == 'apply':
+if stmt[-3][1].upper() == 'APPLY':
 in_batch = False
-elif stmt[0][0] == 'begin':
+elif stmt[0][1].upper() == 'BEGIN':
 in_batch = True
 return output, in_batch
 

[jira] [Commented] (CASSANDRA-10309) Avoid always looking up column type

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742430#comment-14742430
 ] 

Benedict commented on CASSANDRA-10309:
--

It's not the schema metadata we're looking it up in, but the type stored 
against the sstable. There are definitely more performant ways of doing this, 
though.

* For instance, we could set the map to null if in 
{{SerializationHeader.Component.toHeader}} if all of the types match
* We could construct a special {{ColumnDefinition}} type, that contains the 
type we read from the sstable, but also stores the "real" {{ColumnDefinition}} 
as a field; in {{BufferCell.deserialize}} we could then {{instanceof}} check if 
we're one of these, and unwrap the {{ColumnDefinition}} to pass to our 
constructor.

Although I must admit I'm not super familiar with how we handle type changes. 
It looks like we permit type changes that have different representations, but 
don't make any attempt to convert the type

> Avoid always looking up column type
> ---
>
> Key: CASSANDRA-10309
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10309
> Project: Cassandra
>  Issue Type: Bug
>Reporter: T Jake Luciani
>Priority: Minor
> Fix For: 3.x
>
>
> Doing some read profiling I noticed we always seem to look up the type of a 
> column from the schema metadata when we have the type already in the column 
> class.
> This one simple change to SerializationHeader improves read performance 
> non-trivially.
> https://github.com/tjake/cassandra/commit/69b94c389b3f36aa035ac4619fd22d1f62ea80b2
> http://cstar.datastax.com/graph?stats=3fb1ced4-58c7-11e5-9faf-42010af0688f=op_rate=2_read=1_aggregates=true=0=357.94=0=157416.6
> I assume we are looking this up to deal with schema changes. But I'm sure 
> there is a more performant way of doing this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[10/10] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[snazy]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e234b314
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e234b314
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e234b314

Branch: refs/heads/trunk
Commit: e234b314a550df301545feda01647a77453f6b64
Parents: 1e66e97 29c6f1d
Author: Robert Stupp 
Authored: Sun Sep 13 11:37:26 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:37:26 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e234b314/CHANGES.txt
--

[08/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[snazy]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29c6f1d9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29c6f1d9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29c6f1d9

Branch: refs/heads/cassandra-3.0
Commit: 29c6f1d96f07faf5788779416b56eb8b041035b4
Parents: 00264e8 9791796
Author: Robert Stupp 
Authored: Sun Sep 13 11:35:23 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 11:35:58 2015 +0200

--
 CHANGES.txt| 1 +
 pylib/cqlshlib/cql3handling.py | 2 +-
 pylib/cqlshlib/cqlhandling.py  | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/29c6f1d9/CHANGES.txt
--
diff --cc CHANGES.txt
index d367b85,bd24781..bacedaf
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,42 -1,8 +1,43 @@@
 -2.2.2
 +3.0.0-rc1
 + * Add "shadowable" row tombstones to deal with mv timestamp issues 
(CASSANDRA-10261)
 + * CFS.loadNewSSTables() broken for pre-3.0 sstables
 + * Cache selected index in read command to reduce lookups (CASSANDRA-10215)
 + * Small optimizations of sstable index serialization (CASSANDRA-10232)
 + * Support for both encrypted and unencrypted native transport connections 
(CASSANDRA-9590)
 +Merged from 2.2:
   * Defer default role manager setup until all nodes are on 2.2+ 
(CASSANDRA-9761)
 + * Handle missing RoleManager in config after upgrade to 2.2 (CASSANDRA-10209)
 +Merged from 2.1:
++ * BATCH statement is broken in cqlsh (CASSANDRA-10272)
 + * (cqlsh) Make cqlsh PEP8 Compliant (CASSANDRA-10066)
 + * (cqlsh) Fix error when starting cqlsh with --debug (CASSANDRA-10282)
 + * Scrub, Cleanup and Upgrade do not unmark compacting until all operations
 +   have completed, regardless of the occurence of exceptions (CASSANDRA-10274)
 +
 +
 +3.0.0-beta2
 + * Fix columns returned by AbstractBtreePartitions (CASSANDRA-10220)
 + * Fix backward compatibility issue due to AbstractBounds serialization bug 
(CASSANDRA-9857)
 + * Fix startup error when upgrading nodes (CASSANDRA-10136)
 + * Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation 
(CASSANDRA-10147)
 + * Improve batchlog write patch (CASSANDRA-9673)
 + * Re-apply MaterializedView updates on commitlog replay (CASSANDRA-10164)
 + * Require AbstractType.isByteOrderComparable declaration in constructor 
(CASSANDRA-9901)
 + * Avoid digest mismatch on upgrade to 3.0 (CASSANDRA-9554)
 + * Fix Materialized View builder when adding multiple MVs (CASSANDRA-10156)
 + * Choose better poolingOptions for protocol v4 in cassandra-stress 
(CASSANDRA-10182)
 + * Fix LWW bug affecting Materialized Views (CASSANDRA-10197)
 + * Ensures frozen sets and maps are always sorted (CASSANDRA-10162)
 + * Don't deadlock when flushing CFS backed custom indexes (CASSANDRA-10181)
 + * Fix double flushing of secondary index tables (CASSANDRA-10180)
 + * Fix incorrect handling of range tombstones in thrift (CASSANDRA-10046)
 + * Only use batchlog when paired materialized view replica is remote 
(CASSANDRA-10061)
 + * Reuse TemporalRow when updating multiple MaterializedViews 
(CASSANDRA-10060)
 + * Validate gc_grace_seconds for batchlog writes and MVs (CASSANDRA-9917)
 + * Fix sstablerepairedset (CASSANDRA-10132)
 +Merged from 2.2:
   * Cancel transaction for sstables we wont redistribute index summary
 for (CASSANDRA-10270)
 - * Handle missing RoleManager in config after upgrade to 2.2 
(CASSANDRA-10209) 
   * Retry snapshot deletion after compaction and gc on Windows 
(CASSANDRA-10222)
   * Fix failure to start with space in directory path on Windows 
(CASSANDRA-10239)
   * Fix repair hang when snapshot failed (CASSANDRA-10057)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/29c6f1d9/pylib/cqlshlib/cql3handling.py
--

[jira] [Created] (CASSANDRA-10311) It looks like our type alterations may be buggy

[Benedict (JIRA)]

Benedict created CASSANDRA-10311:


 Summary: It looks like our type alterations may be buggy
 Key: CASSANDRA-10311
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10311
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benedict


We should document how type coercion works, in all contexts (UDFs, query 
responses, merging), and what our criteria are for success. Right now it looks 
like we perform no conversion, so we should require that they are compared in 
the same way (if they are clusterings), and that they at least have the same 
number of bytes (if both fixed width).

Integer type considers itself value compatible with Int32 and Long, which from 
an AlterTable point of view at least seems potentially problematic. 

It's very likely I'm missing something. However as it stands we seem able to 
read an old type from an sstable, have it make it through a compaction 
unscathed, and write out the same bytes "as" the new type. If I'm correct about 
this behaviour, this will corrupt this partition in the new sstable so that it 
cannot be read.

Not marking as critical/blocker, as I'm not familiar enough with how this works 
to say if this brief analysis is correct, but if I am we should raise the 
priority.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[snazy]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8564f09c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8564f09c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8564f09c

Branch: refs/heads/trunk
Commit: 8564f09c17f2d2280251031602efbf4b1b3dfc94
Parents: d7d7db9 16497fd
Author: Robert Stupp 
Authored: Sun Sep 13 14:19:39 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 14:19:39 2015 +0200

--

--

[1/3] cassandra git commit: ninja-fix failing LegacySSTableTest

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 4b4d69f77 -> 16497fd93
  refs/heads/trunk d7d7db934 -> 8564f09c1


ninja-fix failing LegacySSTableTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16497fd9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16497fd9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16497fd9

Branch: refs/heads/cassandra-3.0
Commit: 16497fd93f01780e279dcd671f55efb96d3921ef
Parents: 4b4d69f
Author: Robert Stupp 
Authored: Sun Sep 13 14:18:46 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 14:18:46 2015 +0200

--
 .../legacy_ma_clust/ma-1-big-CompressionInfo.db  | Bin 83 -> 83 bytes
 .../legacy_ma_clust/ma-1-big-Data.db | Bin 5032 -> 5045 bytes
 .../legacy_ma_clust/ma-1-big-Digest.crc32|   2 +-
 .../legacy_ma_clust/ma-1-big-Statistics.db   | Bin 7045 -> 7045 bytes
 .../legacy_ma_clust/ma-1-big-TOC.txt |  10 +-
 .../ma-1-big-CompressionInfo.db  | Bin 75 -> 75 bytes
 .../legacy_ma_clust_counter/ma-1-big-Data.db | Bin 4427 -> 4428 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 7054 -> 7054 bytes
 .../legacy_ma_clust_counter/ma-1-big-TOC.txt |  10 +-
 .../legacy_ma_simple/ma-1-big-Data.db| Bin 85 -> 85 bytes
 .../legacy_ma_simple/ma-1-big-Digest.crc32   |   2 +-
 .../legacy_ma_simple/ma-1-big-Statistics.db  | Bin 4598 -> 4598 bytes
 .../legacy_ma_simple/ma-1-big-TOC.txt|  10 +-
 .../legacy_ma_simple_counter/ma-1-big-Data.db| Bin 106 -> 106 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 4607 -> 4607 bytes
 .../legacy_ma_simple_counter/ma-1-big-TOC.txt|  10 +-
 18 files changed, 24 insertions(+), 24 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
index ba05e4b..4955d4b 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db
index 633d0ff..f4ce9a9 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
index 6900022..7eb78a6 100644
--- 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
+++ 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
@@ -1 +1 @@
-1839244592
\ No newline at end of file
+3136403356
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
index 96f0d3a..7fdadbd 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-TOC.txt
--
diff --git 

[jira] [Commented] (CASSANDRA-10266) Introduce direct unit test coverage for Rows

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742467#comment-14742467
 ] 

Benedict commented on CASSANDRA-10266:
--

Is there any chance we could introduce some randomness to this test? I know 
most of the codebase takes this approach of defining a specific test, but I'd 
like to start pushing us in the direction of randomised testing as standard, 
especially for critical pieces of infrastructure like this. Are you confident 
that all possible combinations are explored? Ironically, the fact that you 
caught a bug (which is fantastic, btw) makes me more inclined to request this, 
as it looks like it was dependent on selecting a specific relationship, and 
there may be others we do not think to select.

We could instead generate two (or more) random Rows, and confirm that all of 
the differences are reported (and no more), and just run this many times (or a 
handful, and have a {{LongTest}} equivalent that just runs the same for much 
longer). It would also mean the only difference between each of the testEmpty\* 
etc are just configuration, so make the tests easier to modify. This could most 
likely be reused again for any testing of partition-wide behaviours. WDYT?

> Introduce direct unit test coverage for Rows
> 
>
> Key: CASSANDRA-10266
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10266
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Benedict
>Assignee: Blake Eggleston
> Fix For: 3.0.0 rc1
>
>
> As with much of the codebase, we have no direct unit test coverage for 
> {{Rows}}, and we should remedy this given how central it is to behaviour.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9975) Flatten Iterator call hierarchy with a shared Transformer

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742477#comment-14742477
 ] 

Benedict commented on CASSANDRA-9975:
-

On writing the follow up, CASSANDRA-9976, it became apparent it was easier and 
cleaner to do both at once, since they ultimately share quite a bit of 
functionality and approach. This is now available 
[here|https://github.com/belliottsmith/cassandra/tree/9975]. 

This version also goes a step further to completely eliminate the wrapping 
partition iterators, and all but a handful of wrapping row iterators (the 
remainder of which can be eliminated following CASSANDRA-9986). This is 
achieved by supporting concatenation / short read protection via a 
transformation function that "refils" the contents of an iterator with one or 
more new iterators. i.e.m once the input we're transforming is exhausted, we 
ask for a new one, which is swapped in, and the function stacks updated as 
necessary. This all but eliminates the jumping around of control flow for the 
outer execution of a query, although the inner execution (memtable, sstable) 
still has some cleaning up to be done to make it easy to debug.

> Flatten Iterator call hierarchy with a shared Transformer
> -
>
> Key: CASSANDRA-9975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9975
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.x
>
>
> Stepping through a read response is made exceedingly difficult by the sheer 
> depth of the call hierarchy, and how rapidly your context jumps around. This 
> ticket intend to partially address that, by flattening one of the main causes 
> of this: iterator transformations.
> I have a patch that attempts to mitigate (but not entirely eliminate) this, 
> through the introduction of a {{RowTransformer}} class that all 
> transformations are applied through. If a transformation has already been 
> applied, the {{RowTransformer}} class does not wrap a new iterator, but 
> instead returns a new {{RowTransformer}} that wraps the original underlying 
> (untransformed) iterator and both transformations. This can accumulate an 
> arbitrary number of transformations and, quite importantly, can apply the 
> filtration step {{Unfiltered -> Row}}  in the same instance as well. The 
> intention being that a majority of control flow happens inside this 
> {{RowTransformer}}, so there is far less context jumping to cope with.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-9976) Flatten PartitionIterator call hierarchy with PartitionTransformer

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict resolved CASSANDRA-9976.
-
   Resolution: Duplicate
Fix Version/s: (was: 3.0.x)

Subsumed into CASSANDRA-9975

> Flatten PartitionIterator call hierarchy with PartitionTransformer
> --
>
> Key: CASSANDRA-9976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9976
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
>
> Much like CASSANDRA-9975, except for partitions. This is less involved, and 
> probably less essential, but still helpful. This has been split out to keep 
> the change iterative and piecemiel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: ninja fix ScheduledExecutorService spinning CPU consumption with 0 corePoolSize

[benedict]

ninja fix ScheduledExecutorService spinning CPU consumption with 0 corePoolSize


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b4d69f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b4d69f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b4d69f7

Branch: refs/heads/trunk
Commit: 4b4d69f7786549f33cfd5f71c34b00fef3d6fc91
Parents: 29c6f1d
Author: Benedict Elliott Smith 
Authored: Sun Sep 13 12:05:27 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Sun Sep 13 12:18:52 2015 +0100

--
 src/java/org/apache/cassandra/utils/concurrent/Ref.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b4d69f7/src/java/org/apache/cassandra/utils/concurrent/Ref.java
--
diff --git a/src/java/org/apache/cassandra/utils/concurrent/Ref.java 
b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
index 92c1a0c..856c21e 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
@@ -317,7 +317,7 @@ public final class Ref implements RefCounted
 private static final Set globallyExtant = 
Collections.newSetFromMap(new ConcurrentHashMap<>());
 static final ReferenceQueue referenceQueue = new 
ReferenceQueue<>();
 private static final ExecutorService EXEC = 
Executors.newFixedThreadPool(1, new NamedThreadFactory("Reference-Reaper"));
-private static final ScheduledExecutorService STRONG_LEAK_DETECTOR = 
!DEBUG_ENABLED ? null : Executors.newScheduledThreadPool(0, new 
NamedThreadFactory("Strong-Reference-Leak-Detector"));
+private static final ScheduledExecutorService STRONG_LEAK_DETECTOR = 
!DEBUG_ENABLED ? null : Executors.newScheduledThreadPool(1, new 
NamedThreadFactory("Strong-Reference-Leak-Detector"));
 static
 {
 EXEC.execute(new ReferenceReaper());

[1/3] cassandra git commit: ninja fix ScheduledExecutorService spinning CPU consumption with 0 corePoolSize

[benedict]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 29c6f1d96 -> 4b4d69f77
  refs/heads/trunk e234b314a -> 4f0924c1b


ninja fix ScheduledExecutorService spinning CPU consumption with 0 corePoolSize


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b4d69f7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b4d69f7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b4d69f7

Branch: refs/heads/cassandra-3.0
Commit: 4b4d69f7786549f33cfd5f71c34b00fef3d6fc91
Parents: 29c6f1d
Author: Benedict Elliott Smith 
Authored: Sun Sep 13 12:05:27 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Sun Sep 13 12:18:52 2015 +0100

--
 src/java/org/apache/cassandra/utils/concurrent/Ref.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b4d69f7/src/java/org/apache/cassandra/utils/concurrent/Ref.java
--
diff --git a/src/java/org/apache/cassandra/utils/concurrent/Ref.java 
b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
index 92c1a0c..856c21e 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
@@ -317,7 +317,7 @@ public final class Ref implements RefCounted
 private static final Set globallyExtant = 
Collections.newSetFromMap(new ConcurrentHashMap<>());
 static final ReferenceQueue referenceQueue = new 
ReferenceQueue<>();
 private static final ExecutorService EXEC = 
Executors.newFixedThreadPool(1, new NamedThreadFactory("Reference-Reaper"));
-private static final ScheduledExecutorService STRONG_LEAK_DETECTOR = 
!DEBUG_ENABLED ? null : Executors.newScheduledThreadPool(0, new 
NamedThreadFactory("Strong-Reference-Leak-Detector"));
+private static final ScheduledExecutorService STRONG_LEAK_DETECTOR = 
!DEBUG_ENABLED ? null : Executors.newScheduledThreadPool(1, new 
NamedThreadFactory("Strong-Reference-Leak-Detector"));
 static
 {
 EXEC.execute(new ReferenceReaper());

[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[benedict]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f0924c1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f0924c1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f0924c1

Branch: refs/heads/trunk
Commit: 4f0924c1bf9c5faadddaac7024419bae7bf423fa
Parents: e234b31 4b4d69f
Author: Benedict Elliott Smith 
Authored: Sun Sep 13 12:19:04 2015 +0100
Committer: Benedict Elliott Smith 
Committed: Sun Sep 13 12:19:04 2015 +0100

--
 src/java/org/apache/cassandra/utils/concurrent/Ref.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--

[jira] [Updated] (CASSANDRA-9975) Flatten Iterator call hierarchy with a shared Transformer

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-9975:

Summary: Flatten Iterator call hierarchy with a shared Transformer  (was: 
Flatten RowIterator call hierarchy with a shared RowTransformer)

> Flatten Iterator call hierarchy with a shared Transformer
> -
>
> Key: CASSANDRA-9975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9975
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 3.0.x
>
>
> Stepping through a read response is made exceedingly difficult by the sheer 
> depth of the call hierarchy, and how rapidly your context jumps around. This 
> ticket intend to partially address that, by flattening one of the main causes 
> of this: iterator transformations.
> I have a patch that attempts to mitigate (but not entirely eliminate) this, 
> through the introduction of a {{RowTransformer}} class that all 
> transformations are applied through. If a transformation has already been 
> applied, the {{RowTransformer}} class does not wrap a new iterator, but 
> instead returns a new {{RowTransformer}} that wraps the original underlying 
> (untransformed) iterator and both transformations. This can accumulate an 
> arbitrary number of transformations and, quite importantly, can apply the 
> filtration step {{Unfiltered -> Row}}  in the same instance as well. The 
> intention being that a majority of control flow happens inside this 
> {{RowTransformer}}, so there is far less context jumping to cope with.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10312) Enable custom 2i to opt out of post-streaming rebuilds

[Sam Tunnicliffe (JIRA)]

Sam Tunnicliffe created CASSANDRA-10312:
---

 Summary: Enable custom 2i to opt out of post-streaming rebuilds
 Key: CASSANDRA-10312
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10312
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
 Fix For: 3.0.0 rc1


Custom 2i implementations may not always want to participate in index rebuild 
following streaming operations, as they may prefer to have more control over 
when and how the recieved sstables are indexed.
We could add an {{includeInRebuild()}} method to {{Index}} which would be used 
to filter the set of indexes built in 
{{SecondaryIndexManager#buildAllIndexesBlocking}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10311) It looks like our type alterations may be buggy

[Robert Stupp (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742444#comment-14742444
 ] 

Robert Stupp commented on CASSANDRA-10311:
--

Probably related: CASSANDRA-10027


> It looks like our type alterations may be buggy
> ---
>
> Key: CASSANDRA-10311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10311
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>
> We should document how type coercion works, in all contexts (UDFs, query 
> responses, merging), and what our criteria are for success. Right now it 
> looks like we perform no conversion, so we should require that they are 
> compared in the same way (if they are clusterings), and that they at least 
> have the same number of bytes (if both fixed width).
> Integer type considers itself value compatible with Int32 and Long, which 
> from an AlterTable point of view at least seems potentially problematic. 
> It's very likely I'm missing something. However as it stands we seem able to 
> read an old type from an sstable, have it make it through a compaction 
> unscathed, and write out the same bytes "as" the new type. If I'm correct 
> about this behaviour, this will corrupt this partition in the new sstable so 
> that it cannot be read.
> Not marking as critical/blocker, as I'm not familiar enough with how this 
> works to say if this brief analysis is correct, but if I am we should raise 
> the priority.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10310) avg() on int give incorrect results (returning int instead of decimal or float)

[Brett Snyder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brett Snyder updated CASSANDRA-10310:
-
Attachment: cassandra-3.0-10310.txt

> avg() on int give incorrect results (returning int instead of decimal or 
> float)
> ---
>
> Key: CASSANDRA-10310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10310
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jon Haddad
>  Labels: patch
> Fix For: 3.x, 2.2.x
>
> Attachments: cassandra-2.2-10310.txt, cassandra-3.0-10310.txt
>
>
> When selecting an avg() of int values, the type of the avg value returned is 
> an int as well, meaning it's rounded off to an incorrect answer.  This is 
> both incorrect and inconsistent with other databases.
> Example:
> {quote}
> cqlsh:test> select * from monkey where id = 1;
>  id | i | v
> +---+---
>   1 | 1 | 1
>   1 | 2 | 1
>   1 | 3 | 2
> (3 rows)
> cqlsh:test> select avg(v) from monkey where id = 1;
>  system.avg(v)
> ---
>  1
> (1 rows)
> {quote}
> I tried avg() with MySQL, here's the result:
> {quote}
> mysql> create table blah ( id int primary key, v int );
> Query OK, 0 rows affected (0.15 sec)
> mysql> insert into blah set id = 1, v = 1;
> Query OK, 1 row affected (0.02 sec)
> mysql> insert into blah set id = 1, v = 1;
> ERROR 1062 (23000): Duplicate entry '1' for key 'PRIMARY'
> mysql> insert into blah set id = 2, v = 1;
> Query OK, 1 row affected (0.01 sec)
> mysql> insert into blah set id = 3, v = 2;
> Query OK, 1 row affected (0.01 sec)
> mysql> select avg(v) from blah;
> ++
> | avg(v) |
> ++
> | 1. |
> ++
> 1 row in set (0.00 sec)
> {quote}
> I created a new table using the above query. The result:
> {quote}
> mysql> create table foo as select avg(v) as a from blah;
> Query OK, 1 row affected, 1 warning (0.04 sec)
> Records: 1  Duplicates: 0  Warnings: 1
> mysql> desc foo;
> +---+---+--+-+-+---+
> | Field | Type  | Null | Key | Default | Extra |
> +---+---+--+-+-+---+
> | a | decimal(14,4) | YES  | | NULL|   |
> +---+---+--+-+-+---+
> 1 row in set (0.01 sec)
> {quote}
> It works the same way in postgres, and to my knowledge, every RDBMs.
> Broken in 2.2, 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10311) It looks like our type alterations may be buggy

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742452#comment-14742452
 ] 

Benedict commented on CASSANDRA-10311:
--

So, taking that as evidence that the problem is real, it looks like at the very 
least we can convert a {{varint}} to a {{date}} and corrupt the serialization. 
I suspect this only occurs because {{IntegerType}} was mistaken for 
{{Int32Type}}, and we should consider renaming it to {{VarIntType}} (and adding 
an IntegerType that extends this, and is deprecated).

Relatedly: what is {{DateType}} used for? It doesn't seem to be exposed 
externally. I assume it is deprecated, but there are no comments about it, or 
its historical contract and how it has been replaced (nor is it marked 
{{@Deprecated}}).

It looks to me like we generally need to do a reconciliation of all of our 
typing behaviours, and preferably document them in a package level javadoc.

> It looks like our type alterations may be buggy
> ---
>
> Key: CASSANDRA-10311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10311
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>
> We should document how type coercion works, in all contexts (UDFs, query 
> responses, merging), and what our criteria are for success. Right now it 
> looks like we perform no conversion, so we should require that they are 
> compared in the same way (if they are clusterings), and that they at least 
> have the same number of bytes (if both fixed width).
> Integer type considers itself value compatible with Int32 and Long, which 
> from an AlterTable point of view at least seems potentially problematic. 
> It's very likely I'm missing something. However as it stands we seem able to 
> read an old type from an sstable, have it make it through a compaction 
> unscathed, and write out the same bytes "as" the new type. If I'm correct 
> about this behaviour, this will corrupt this partition in the new sstable so 
> that it cannot be read.
> Not marking as critical/blocker, as I'm not familiar enough with how this 
> works to say if this brief analysis is correct, but if I am we should raise 
> the priority.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: ninja-fix failing LegacySSTableTest

[snazy]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4f0924c1b -> d7d7db934


ninja-fix failing LegacySSTableTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed67154b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed67154b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed67154b

Branch: refs/heads/trunk
Commit: ed67154b897d7b9745de8f4a1888009266a67f22
Parents: 29c6f1d
Author: Robert Stupp 
Authored: Sun Sep 13 14:13:40 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 14:13:40 2015 +0200

--
 .../legacy_ma_clust/ma-1-big-CompressionInfo.db  | Bin 83 -> 83 bytes
 .../legacy_ma_clust/ma-1-big-Data.db | Bin 5032 -> 5045 bytes
 .../legacy_ma_clust/ma-1-big-Digest.crc32|   2 +-
 .../legacy_ma_clust/ma-1-big-Statistics.db   | Bin 7045 -> 7045 bytes
 .../legacy_ma_clust/ma-1-big-TOC.txt |   8 
 .../ma-1-big-CompressionInfo.db  | Bin 75 -> 75 bytes
 .../legacy_ma_clust_counter/ma-1-big-Data.db | Bin 4427 -> 4447 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 7054 -> 7054 bytes
 .../legacy_ma_clust_counter/ma-1-big-TOC.txt |   8 
 .../legacy_ma_simple/ma-1-big-Data.db| Bin 85 -> 85 bytes
 .../legacy_ma_simple/ma-1-big-Digest.crc32   |   2 +-
 .../legacy_ma_simple/ma-1-big-Statistics.db  | Bin 4598 -> 4598 bytes
 .../legacy_ma_simple/ma-1-big-TOC.txt|   8 
 .../legacy_ma_simple_counter/ma-1-big-Data.db| Bin 106 -> 106 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 4607 -> 4607 bytes
 .../legacy_ma_simple_counter/ma-1-big-TOC.txt|   8 
 18 files changed, 20 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed67154b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
index ba05e4b..f820f8f 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed67154b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db
index 633d0ff..a6ea8e7 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed67154b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
index 6900022..da2f2b3 100644
--- 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
+++ 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
@@ -1 +1 @@
-1839244592
\ No newline at end of file
+923173075
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed67154b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
index 96f0d3a..94ab786 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed67154b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-TOC.txt
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-TOC.txt 

[2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk

[snazy]

Merge branch 'cassandra-3.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d7d7db93
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d7d7db93
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d7d7db93

Branch: refs/heads/trunk
Commit: d7d7db934085566acbd6f15fda3ccab324ab51c0
Parents: 4f0924c ed67154
Author: Robert Stupp 
Authored: Sun Sep 13 14:15:25 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 14:15:25 2015 +0200

--
 .../legacy_ma_clust/ma-1-big-CompressionInfo.db  | Bin 83 -> 83 bytes
 .../legacy_ma_clust/ma-1-big-Data.db | Bin 5032 -> 5045 bytes
 .../legacy_ma_clust/ma-1-big-Digest.crc32|   2 +-
 .../legacy_ma_clust/ma-1-big-Statistics.db   | Bin 7045 -> 7045 bytes
 .../legacy_ma_clust/ma-1-big-TOC.txt |   8 
 .../ma-1-big-CompressionInfo.db  | Bin 75 -> 75 bytes
 .../legacy_ma_clust_counter/ma-1-big-Data.db | Bin 4427 -> 4447 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 7054 -> 7054 bytes
 .../legacy_ma_clust_counter/ma-1-big-TOC.txt |   8 
 .../legacy_ma_simple/ma-1-big-Data.db| Bin 85 -> 85 bytes
 .../legacy_ma_simple/ma-1-big-Digest.crc32   |   2 +-
 .../legacy_ma_simple/ma-1-big-Statistics.db  | Bin 4598 -> 4598 bytes
 .../legacy_ma_simple/ma-1-big-TOC.txt|   8 
 .../legacy_ma_simple_counter/ma-1-big-Data.db| Bin 106 -> 106 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 4607 -> 4607 bytes
 .../legacy_ma_simple_counter/ma-1-big-TOC.txt|   8 
 18 files changed, 20 insertions(+), 20 deletions(-)
--

[jira] [Commented] (CASSANDRA-10249) Make buffered read size configurable

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742483#comment-14742483
 ] 

Benedict commented on CASSANDRA-10249:
--

[~jimplush]: in the meantime, I recommend you switch to using {{mmap}} file 
access, which should resolve the problem for you

> Make buffered read size configurable
> 
>
> Key: CASSANDRA-10249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10249
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Albert P Tobey
> Fix For: 2.1.x
>
> Attachments: Screenshot 2015-09-11 09.32.04.png, Screenshot 
> 2015-09-11 09.34.10.png, patched-2.1.9-dstat-lvn10.png, 
> stock-2.1.9-dstat-lvn10.png, yourkit-screenshot.png
>
>
> On read workloads, Cassandra 2.1 reads drastically more data than it emits 
> over the network. This causes problems throughput the system by wasting disk 
> IO and causing unnecessary GC.
> I have reproduce the issue on clusters and locally with a single instance. 
> The only requirement to reproduce the issue is enough data to blow through 
> the page cache. The default schema and data size with cassandra-stress is 
> sufficient for exposing the issue.
> With stock 2.1.9 I regularly observed anywhere from 300:1  to 500 
> disk:network ratio. That is to say, for 1MB/s of network IO, Cassandra was 
> doing 300-500MB/s of disk reads, saturating the drive.
> After applying this patch for standard IO mode 
> https://gist.github.com/tobert/10c307cf3709a585a7cf the ratio fell to around 
> 100:1 on my local test rig. Latency improved considerably and GC became a lot 
> less frequent.
> I tested with 512 byte reads as well, but got the same performance, which 
> makes sense since all HDD and SSD made in the last few years have a 4K block 
> size (many of them lie and say 512).
> I'm re-running the numbers now and will post them tomorrow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10310) avg() on int give incorrect results (returning int instead of decimal or float)

[Brett Snyder (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brett Snyder updated CASSANDRA-10310:
-
Attachment: cassandra-2.2-10310.txt

> avg() on int give incorrect results (returning int instead of decimal or 
> float)
> ---
>
> Key: CASSANDRA-10310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10310
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jon Haddad
>  Labels: patch
> Fix For: 3.x, 2.2.x
>
> Attachments: cassandra-2.2-10310.txt
>
>
> When selecting an avg() of int values, the type of the avg value returned is 
> an int as well, meaning it's rounded off to an incorrect answer.  This is 
> both incorrect and inconsistent with other databases.
> Example:
> {quote}
> cqlsh:test> select * from monkey where id = 1;
>  id | i | v
> +---+---
>   1 | 1 | 1
>   1 | 2 | 1
>   1 | 3 | 2
> (3 rows)
> cqlsh:test> select avg(v) from monkey where id = 1;
>  system.avg(v)
> ---
>  1
> (1 rows)
> {quote}
> I tried avg() with MySQL, here's the result:
> {quote}
> mysql> create table blah ( id int primary key, v int );
> Query OK, 0 rows affected (0.15 sec)
> mysql> insert into blah set id = 1, v = 1;
> Query OK, 1 row affected (0.02 sec)
> mysql> insert into blah set id = 1, v = 1;
> ERROR 1062 (23000): Duplicate entry '1' for key 'PRIMARY'
> mysql> insert into blah set id = 2, v = 1;
> Query OK, 1 row affected (0.01 sec)
> mysql> insert into blah set id = 3, v = 2;
> Query OK, 1 row affected (0.01 sec)
> mysql> select avg(v) from blah;
> ++
> | avg(v) |
> ++
> | 1. |
> ++
> 1 row in set (0.00 sec)
> {quote}
> I created a new table using the above query. The result:
> {quote}
> mysql> create table foo as select avg(v) as a from blah;
> Query OK, 1 row affected, 1 warning (0.04 sec)
> Records: 1  Duplicates: 0  Warnings: 1
> mysql> desc foo;
> +---+---+--+-+-+---+
> | Field | Type  | Null | Key | Default | Extra |
> +---+---+--+-+-+---+
> | a | decimal(14,4) | YES  | | NULL|   |
> +---+---+--+-+-+---+
> 1 row in set (0.01 sec)
> {quote}
> It works the same way in postgres, and to my knowledge, every RDBMs.
> Broken in 2.2, 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: ninja-fix failing LegacySSTableTest

[snazy]

ninja-fix failing LegacySSTableTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16497fd9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16497fd9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16497fd9

Branch: refs/heads/trunk
Commit: 16497fd93f01780e279dcd671f55efb96d3921ef
Parents: 4b4d69f
Author: Robert Stupp 
Authored: Sun Sep 13 14:18:46 2015 +0200
Committer: Robert Stupp 
Committed: Sun Sep 13 14:18:46 2015 +0200

--
 .../legacy_ma_clust/ma-1-big-CompressionInfo.db  | Bin 83 -> 83 bytes
 .../legacy_ma_clust/ma-1-big-Data.db | Bin 5032 -> 5045 bytes
 .../legacy_ma_clust/ma-1-big-Digest.crc32|   2 +-
 .../legacy_ma_clust/ma-1-big-Statistics.db   | Bin 7045 -> 7045 bytes
 .../legacy_ma_clust/ma-1-big-TOC.txt |  10 +-
 .../ma-1-big-CompressionInfo.db  | Bin 75 -> 75 bytes
 .../legacy_ma_clust_counter/ma-1-big-Data.db | Bin 4427 -> 4428 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 7054 -> 7054 bytes
 .../legacy_ma_clust_counter/ma-1-big-TOC.txt |  10 +-
 .../legacy_ma_simple/ma-1-big-Data.db| Bin 85 -> 85 bytes
 .../legacy_ma_simple/ma-1-big-Digest.crc32   |   2 +-
 .../legacy_ma_simple/ma-1-big-Statistics.db  | Bin 4598 -> 4598 bytes
 .../legacy_ma_simple/ma-1-big-TOC.txt|  10 +-
 .../legacy_ma_simple_counter/ma-1-big-Data.db| Bin 106 -> 106 bytes
 .../ma-1-big-Digest.crc32|   2 +-
 .../ma-1-big-Statistics.db   | Bin 4607 -> 4607 bytes
 .../legacy_ma_simple_counter/ma-1-big-TOC.txt|  10 +-
 18 files changed, 24 insertions(+), 24 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
index ba05e4b..4955d4b 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-CompressionInfo.db
 differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db
index 633d0ff..f4ce9a9 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Data.db 
differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
index 6900022..7eb78a6 100644
--- 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
+++ 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Digest.crc32
@@ -1 +1 @@
-1839244592
\ No newline at end of file
+3136403356
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
index 96f0d3a..7fdadbd 100644
Binary files 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 and 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-Statistics.db
 differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/16497fd9/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-TOC.txt
--
diff --git 
a/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-TOC.txt 
b/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_clust/ma-1-big-TOC.txt
index 00915e9..a6eda71 100644
--- 

[jira] [Assigned] (CASSANDRA-9739) Migrate counter-cache to be fully off-heap

[Robert Stupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp reassigned CASSANDRA-9739:
---

Assignee: Robert Stupp

> Migrate counter-cache to be fully off-heap
> --
>
> Key: CASSANDRA-9739
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9739
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.x
>
>
> Counter cache still uses a concurrent map on-heap. This could go to off-heap 
> and feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure 
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some 
> benefit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9921) Combine MV schema definition with MV table definition

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742669#comment-14742669
 ] 

Aleksey Yeschenko commented on CASSANDRA-9921:
--

[~aholmber] Added {{base_table_name}} and force-pushed to the branch, drivers 
should be unblocked now.

[~carlyeks] I pushed some changes 
[here|https://github.com/iamaleksey/cassandra/commits/9921-3.0], including 
Tyler's unit test, that is currently failing. Can you make it pass? Thanks.

> Combine MV schema definition with MV table definition
> -
>
> Key: CASSANDRA-9921
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9921
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
>  Labels: client-impacting, materializedviews
> Fix For: 3.0.0 rc1
>
> Attachments: 9921-unit-test.txt
>
>
> Prevent MV from reusing {{system_schema.tables}} and instead move those 
> properties into the {{system_schema.materializedviews}} table to keep them 
> separate entities.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742670#comment-14742670
 ] 

Benedict commented on CASSANDRA-9738:
-

bq. In a lot of cases the RIE is going to be enclosed by a Closable iterator 
anyways.

Given our historically poor handling of resource release, and how many nooks 
and crannies they end up in now, I'd prefer we didn't depend on the closure of 
iterators until we have some better facilities in place for catching leaks, 
like CASSANDRA-9918. Right now I don't think we're likely to leak much on a 
per-request/leak basis (just prevent some shared cleanup, per discarded major 
resource, such as an sstable) if we fail to close an iterator. So this could 
magnify the effect of any such bug.

> Migrate key-cache to be fully off-heap
> --
>
> Key: CASSANDRA-9738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> Key cache still uses a concurrent map on-heap. This could go to off-heap and 
> feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure 
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some 
> benefit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-10310) avg() on int give incorrect results (returning int instead of decimal or float)

[Benjamin Lerer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer resolved CASSANDRA-10310.

   Resolution: Invalid
Fix Version/s: (was: 2.2.x)
   (was: 3.x)

> avg() on int give incorrect results (returning int instead of decimal or 
> float)
> ---
>
> Key: CASSANDRA-10310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10310
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jon Haddad
>  Labels: patch
> Attachments: cassandra-2.2-10310.txt, cassandra-3.0-10310.txt
>
>
> When selecting an avg() of int values, the type of the avg value returned is 
> an int as well, meaning it's rounded off to an incorrect answer.  This is 
> both incorrect and inconsistent with other databases.
> Example:
> {quote}
> cqlsh:test> select * from monkey where id = 1;
>  id | i | v
> +---+---
>   1 | 1 | 1
>   1 | 2 | 1
>   1 | 3 | 2
> (3 rows)
> cqlsh:test> select avg(v) from monkey where id = 1;
>  system.avg(v)
> ---
>  1
> (1 rows)
> {quote}
> I tried avg() with MySQL, here's the result:
> {quote}
> mysql> create table blah ( id int primary key, v int );
> Query OK, 0 rows affected (0.15 sec)
> mysql> insert into blah set id = 1, v = 1;
> Query OK, 1 row affected (0.02 sec)
> mysql> insert into blah set id = 1, v = 1;
> ERROR 1062 (23000): Duplicate entry '1' for key 'PRIMARY'
> mysql> insert into blah set id = 2, v = 1;
> Query OK, 1 row affected (0.01 sec)
> mysql> insert into blah set id = 3, v = 2;
> Query OK, 1 row affected (0.01 sec)
> mysql> select avg(v) from blah;
> ++
> | avg(v) |
> ++
> | 1. |
> ++
> 1 row in set (0.00 sec)
> {quote}
> I created a new table using the above query. The result:
> {quote}
> mysql> create table foo as select avg(v) as a from blah;
> Query OK, 1 row affected, 1 warning (0.04 sec)
> Records: 1  Duplicates: 0  Warnings: 1
> mysql> desc foo;
> +---+---+--+-+-+---+
> | Field | Type  | Null | Key | Default | Extra |
> +---+---+--+-+-+---+
> | a | decimal(14,4) | YES  | | NULL|   |
> +---+---+--+-+-+---+
> 1 row in set (0.01 sec)
> {quote}
> It works the same way in postgres, and to my knowledge, every RDBMs.
> Broken in 2.2, 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10310) avg() on int give incorrect results (returning int instead of decimal or float)

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742604#comment-14742604
 ] 

Benjamin Lerer commented on CASSANDRA-10310:


{quote}It works the same way in postgres, and to my knowledge, every 
RDBMs.{quote}
Both Oracle and Microsoft SQL server return an {{int}} if the input type is an 
{{int}}.
*[Oracle 
documentation|http://docs.oracle.com/cd/E11882_01/server.112/e41084/functions018.htm#SQLRF00609]
*[Transact-SQL 
documentation|https://msdn.microsoft.com/en-us/library/ms177677.aspx]

If you want to read the full discussion about aggregates return types, you can 
found it 
[here|https://issues.apache.org/jira/browse/CASSANDRA-4914?focusedCommentId=14132148=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14132148].

> avg() on int give incorrect results (returning int instead of decimal or 
> float)
> ---
>
> Key: CASSANDRA-10310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10310
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Jon Haddad
>  Labels: patch
> Fix For: 3.x, 2.2.x
>
> Attachments: cassandra-2.2-10310.txt, cassandra-3.0-10310.txt
>
>
> When selecting an avg() of int values, the type of the avg value returned is 
> an int as well, meaning it's rounded off to an incorrect answer.  This is 
> both incorrect and inconsistent with other databases.
> Example:
> {quote}
> cqlsh:test> select * from monkey where id = 1;
>  id | i | v
> +---+---
>   1 | 1 | 1
>   1 | 2 | 1
>   1 | 3 | 2
> (3 rows)
> cqlsh:test> select avg(v) from monkey where id = 1;
>  system.avg(v)
> ---
>  1
> (1 rows)
> {quote}
> I tried avg() with MySQL, here's the result:
> {quote}
> mysql> create table blah ( id int primary key, v int );
> Query OK, 0 rows affected (0.15 sec)
> mysql> insert into blah set id = 1, v = 1;
> Query OK, 1 row affected (0.02 sec)
> mysql> insert into blah set id = 1, v = 1;
> ERROR 1062 (23000): Duplicate entry '1' for key 'PRIMARY'
> mysql> insert into blah set id = 2, v = 1;
> Query OK, 1 row affected (0.01 sec)
> mysql> insert into blah set id = 3, v = 2;
> Query OK, 1 row affected (0.01 sec)
> mysql> select avg(v) from blah;
> ++
> | avg(v) |
> ++
> | 1. |
> ++
> 1 row in set (0.00 sec)
> {quote}
> I created a new table using the above query. The result:
> {quote}
> mysql> create table foo as select avg(v) as a from blah;
> Query OK, 1 row affected, 1 warning (0.04 sec)
> Records: 1  Duplicates: 0  Warnings: 1
> mysql> desc foo;
> +---+---+--+-+-+---+
> | Field | Type  | Null | Key | Default | Extra |
> +---+---+--+-+-+---+
> | a | decimal(14,4) | YES  | | NULL|   |
> +---+---+--+-+-+---+
> 1 row in set (0.01 sec)
> {quote}
> It works the same way in postgres, and to my knowledge, every RDBMs.
> Broken in 2.2, 3.0.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10198) 3.0 hints should be streamed on decomission

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742703#comment-14742703
 ] 

Aleksey Yeschenko commented on CASSANDRA-10198:
---

1. If we were to continue using an {{Iterable}} there, then the Java 8 
way to deal with streams would be to use a collector ({{Collectors.toList()}} 
in this case)
2. However, in Java 8, we should use a {{CompletableFuture}} here, and not an 
{{Iterable}} of {{Future}}, or a single future.
3. We are picking just one destination for hints. Which is fine, I guess. Yet 
we potentially use multiple threads to deliver to the same node, and thus 
disrespect the configured rate limit, which is per-destination. These should be 
serialized.
4. Failure to deliver is completely ignored. We must deliver all hints before 
returning, as this has implications for batchlog-generated hints.
5. Please add some documentation for the method. On the level of {{excise}} 
javadoc at least.

> 3.0 hints should be streamed on decomission
> ---
>
> Key: CASSANDRA-10198
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10198
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Marcus Eriksson
> Fix For: 3.0.0 rc1
>
>
> CASSANDRA-6230 added all the necessary pieces in the initial release, but 
> streaming itself didn't make it in time.
> Now that hints are stored in flat files, we cannot just stream hints 
> sstables. Instead we need to handoff hints files.
> Essentially we need to rewrite {{StorageService::streamHints}} to be 
> CASSANDRA-6230 aware.
> {{HintMessage}} and {{HintVerbHandler}} can already handle hints targeted for 
> other nodes (see javadoc for both, it's documented reasonably).
> {{HintsDispatcher}} also takes hostId as an argument, and can stream any 
> hints to any nodes.
> The building blocks are all there - we just need 
> {{StorageService::streamHints}} to pick the optimal candidate for each file 
> and use {{HintsDispatcher}} to stream the files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10272) BATCH statement is broken in cqlsh

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742731#comment-14742731
 ] 

Stefania commented on CASSANDRA-10272:
--

Thanks Robert. I've created CASSANDRA-10313 for the unrelated failure.

> BATCH statement is broken in cqlsh
> --
>
> Key: CASSANDRA-10272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10272
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: CentOS 7.1/x64
>Reporter: Vovodroid
>Assignee: Stefania
> Fix For: 3.0.0 rc1, 2.1.10, 2.2.2
>
> Attachments: 10272-2.1.txt
>
>
> BEGIN BATCH  APPLY BATCH is not parsed correctly.
> Steps:
> {code}
> CREATE KEYSPACE Excelsior  WITH 
> REPLICATION={'class':'SimpleStrategy','replication_factor':1};
> CREATE TABLE excelsior.data (id int primary key);
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0); APPLY BATCH ;
> {code}
> Error
> {code}
> SyntaxException:  message="line 0:-1 mismatched input '' expecting K_APPLY">
> SyntaxException:  message="line 1:0 no viable alternative at input 'APPLY' ([APPLY]...)">
> {code}
> While 
> {code}
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0)  APPLY BATCH ;
> {code}
> without *;* after insert works.
> Consequently neither
> {code}
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0);INSERT INTO 
> excelsior.data (id) VALUES (0); APPLY BATCH ;
> {code}
> Error:
> {code}
> SyntaxException:  message="line 0:-1 mismatched input '' expecting K_APPLY">
> SyntaxException:  message="line 1:0 no viable alternative at input 'APPLY' ([APPLY]...)">
> {code}
> nor
> {code}
> BEGIN BATCH INSERT INTO excelsior.data (id) VALUES (0);INSERT INTO 
> excelsior.data (id) VALUES (0) APPLY BATCH ;
> {code}
> Error
> {code}
> SyntaxException:  message="line 0:-1 mismatched input '' expecting K_APPLY">
> SyntaxException:  message="line 1:43 missing EOF at 'APPLY' (...(id) VALUES (0) [APPLY] 
> BATCH...)">
> {code}
> works.
> It was OK in 2.2.0 and 3.0 beta 1.
> 3.0-beta2-tentative also affected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742700#comment-14742700
 ] 

Ariel Weisberg commented on CASSANDRA-9738:
---

OK, well I don't have an answer for 3.0 on how to completely eliminate the risk 
that the copying overhead won't be an issue.

If it keeps changing there is no way I will get to the point where I can +1 the 
performance dimension. I might be able to get to +1 correctness if things stop 
changing long enough for me to checkout it out and walk through everything. 
Performance could very well be good enough/better for realistically sized 
indexed entries. If we benchmark the final code and it's better then I guess I 
could be +1 on performance and ignore the overhead of allocating and copying 
for each cache access.

In hindsight wholesale replacing the existing key cache was not a good 
incremental way to effect this change in a low risk way. Introducing a new key 
cache and iterating on it would have allowed for faster development, more 
frequent merging to trunk, smaller reviews etc. We would also always have 
something to ship once release time rolls around even if something change was 
pending.

* [IndexEntryStats, I don't get why you are casting down to int, why not 
checked cast on the way 
out?|https://github.com/apache/cassandra/commit/5d53c29ceb915b1b95bca0fff15b1591d508e8c8#diff-75146ba408a51071a0b19ffdfbb2bb3cR1434]
* [I don't understand overflow protection here. It's a long is it really going 
to 
overflow?|https://github.com/apache/cassandra/commit/5d53c29ceb915b1b95bca0fff15b1591d508e8c8#diff-75146ba408a51071a0b19ffdfbb2bb3cR1434]
* Average not taking into account the distribution is suspect to me. Should the 
allocation be slightly larger than average to prevent reallocation?
* When allocating the offset buffer it's using the count of index infos without 
multiplying by their size?
* We are writing sampled records. I am skeptical this is worth optimizing when 
we have to get this done and correct. There are better options in subsequent 
versions like appending to a file.
* Did this change have an impact and improve some metric?


> Migrate key-cache to be fully off-heap
> --
>
> Key: CASSANDRA-9738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> Key cache still uses a concurrent map on-heap. This could go to off-heap and 
> feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure 
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some 
> benefit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8805) runWithCompactionsDisabled only cancels compactions, which is not the only source of markCompacted

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742717#comment-14742717
 ] 

Benedict commented on CASSANDRA-8805:
-

Thanks. I've pushed a version 
[here|https://github.com/belliottsmith/cassandra/tree/8805], which just cleans 
up ever so slightly.

As a note for the future, it's very helpful for review to avoid squashing until 
after we're ready to commit (or we reach some mutually agreed way point)

> runWithCompactionsDisabled only cancels compactions, which is not the only 
> source of markCompacted
> --
>
> Key: CASSANDRA-8805
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8805
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Benedict
>Assignee: Carl Yeksigian
> Fix For: 2.1.x
>
> Attachments: 8805-2.1.txt
>
>
> Operations like repair that may operate over all sstables cancel compactions 
> before beginning, and fail if there are any files marked compacting after 
> doing so. Redistribution of index summaries is not a compaction, so is not 
> cancelled by this action, but does mark sstables as compacting, so such an 
> action will fail to initiate if there is an index summary redistribution in 
> progress. It seems that IndexSummaryManager needs to register itself as 
> interruptible along with compactions (AFAICT no other actions that may 
> markCompacting are not themselves compactions).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742716#comment-14742716
 ] 

Benedict commented on CASSANDRA-9738:
-

bq. In hindsight wholesale replacing the existing key cache was not a good 
incremental way to effect this change in a low risk way

So, looking at the latest performance comparisons, it doesn't look like the 
benefit is so uniformly profound as to be worth rushing this out. It looks like 
whatever was particularly affecting 3.0 in the early benchmarks (that notably 
wasn't affecting prior versions) has been ironed out. 

If the code is still being majorly revised so close to RC, we should consider 
pushing this back, and if incorporated before GA at least consider making it 
optional. It's turning into a patch of really significant size, and we already 
have a great deal of risk associated with this release.


> Migrate key-cache to be fully off-heap
> --
>
> Key: CASSANDRA-9738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9738
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0.0 rc1
>
>
> Key cache still uses a concurrent map on-heap. This could go to off-heap and 
> feels doable now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure 
> off-heap counter cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some 
> benefit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10313) cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_all_datatypes_read fails locally

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742730#comment-14742730
 ] 

Stefania commented on CASSANDRA-10313:
--

This was discovered whilst testing CASSANDRA-10272.

> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_all_datatypes_read fails 
> locally
> 
>
> Key: CASSANDRA-10313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10313
> Project: Cassandra
>  Issue Type: Test
> Environment: Linux
>Reporter: Stefania
>Assignee: Stefania
>
> I get this failure on my box with TZ at GMT+08:
> {code}
> ==
> FAIL: test_all_datatypes_read (cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/stefi/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
> 674, in test_all_datatypes_read
> self.assertCsvResultEqual(self.tempfile.name, results)
>   File 
> "/home/stefi/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", line 
> 137, in assertCsvResultEqual
> raise e
> AssertionError: Element counts were not equal:
> First has 1, Second has 0:  ['ascii', '1099511627776', '0xbeef', 'True', 
> '3.140124344978758017532527446746826171875', '2.444', '1.1', 
> '127.0.0.1', '25', 
> '\xe3\x83\xbd(\xc2\xb4\xe3\x83\xbc\xef\xbd\x80)\xe3\x83\x8e', '2005-07-14 
> 12:30:00', '30757c2c-584a-11e5-b2d0-9cebe804ecbe', 
> '2471e7de-41e4-478f-a402-e99ed779be76', 'asdf', '36893488147419103232']
> First has 0, Second has 1:  ['ascii', '1099511627776', '0xbeef', 'True', 
> '3.140124344978758017532527446746826171875', '2.444', '1.1', 
> '127.0.0.1', '25', 
> '\xe3\x83\xbd(\xc2\xb4\xe3\x83\xbc\xef\xbd\x80)\xe3\x83\x8e', '2005-07-14 
> 04:30:00', '30757c2c-584a-11e5-b2d0-9cebe804ecbe', 
> '2471e7de-41e4-478f-a402-e99ed779be76', 'asdf', '36893488147419103232']
>  >> begin captured logging << 
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-8VAvBl
> dtest: DEBUG: Importing from csv file: /tmp/tmpGwW8yB
> dtest: WARNING: Mismatch at index: 10
> dtest: WARNING: Value in csv: 2005-07-14 12:30:00
> dtest: WARNING: Value in result: 2005-07-14 04:30:00
> - >> end captured logging << -
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-10313) cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_all_datatypes_read fails locally

[Stefania (JIRA)]

Stefania created CASSANDRA-10313:


 Summary: 
cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest.test_all_datatypes_read fails locally
 Key: CASSANDRA-10313
 URL: https://issues.apache.org/jira/browse/CASSANDRA-10313
 Project: Cassandra
  Issue Type: Test
 Environment: Linux
Reporter: Stefania
Assignee: Stefania


I get this failure on my box with TZ at GMT+08:
{code}
==
FAIL: test_all_datatypes_read (cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest)
--
Traceback (most recent call last):
  File "/home/stefi/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
line 674, in test_all_datatypes_read
self.assertCsvResultEqual(self.tempfile.name, results)
  File "/home/stefi/git/cstar/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py", 
line 137, in assertCsvResultEqual
raise e
AssertionError: Element counts were not equal:
First has 1, Second has 0:  ['ascii', '1099511627776', '0xbeef', 'True', 
'3.140124344978758017532527446746826171875', '2.444', '1.1', 
'127.0.0.1', '25', 
'\xe3\x83\xbd(\xc2\xb4\xe3\x83\xbc\xef\xbd\x80)\xe3\x83\x8e', '2005-07-14 
12:30:00', '30757c2c-584a-11e5-b2d0-9cebe804ecbe', 
'2471e7de-41e4-478f-a402-e99ed779be76', 'asdf', '36893488147419103232']
First has 0, Second has 1:  ['ascii', '1099511627776', '0xbeef', 'True', 
'3.140124344978758017532527446746826171875', '2.444', '1.1', 
'127.0.0.1', '25', 
'\xe3\x83\xbd(\xc2\xb4\xe3\x83\xbc\xef\xbd\x80)\xe3\x83\x8e', '2005-07-14 
04:30:00', '30757c2c-584a-11e5-b2d0-9cebe804ecbe', 
'2471e7de-41e4-478f-a402-e99ed779be76', 'asdf', '36893488147419103232']
 >> begin captured logging << 
dtest: DEBUG: cluster ccm directory: /tmp/dtest-8VAvBl
dtest: DEBUG: Importing from csv file: /tmp/tmpGwW8yB
dtest: WARNING: Mismatch at index: 10
dtest: WARNING: Value in csv: 2005-07-14 12:30:00
dtest: WARNING: Value in result: 2005-07-14 04:30:00
- >> end captured logging << -
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9758) nodetool compactionhistory NPE

[Mark Manley (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742695#comment-14742695
 ] 

Mark Manley commented on CASSANDRA-9758:


This seems broken in 2.2.1, FYI:

{code}
$ nodetool compactionhistory
Compaction History:
error: null
-- StackTrace --
java.lang.NullPointerException
at com.google.common.base.Joiner$MapJoiner.join(Joiner.java:330)
at org.apache.cassandra.utils.FBUtilities.toString(FBUtilities.java:477)
at 
org.apache.cassandra.db.compaction.CompactionHistoryTabularData.from(CompactionHistoryTabularData.java:78)
at 
org.apache.cassandra.db.SystemKeyspace.getCompactionHistory(SystemKeyspace.java:425)
at 
org.apache.cassandra.db.compaction.CompactionManager.getCompactionHistory(CompactionManager.java:1492)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at 
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at 
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at 
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at 
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at 
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at 
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1443)
at 
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at 
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1307)
at 
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1399)
at 
javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:637)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at 
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$241(TCPTransport.java:683)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$1/1383269057.run(Unknown
 Source)
at java.security.AccessController.doPrivileged(Native Method)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

> nodetool compactionhistory NPE
> --
>
> Key: CASSANDRA-9758
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9758
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Pierre N.
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 0001-fix-npe-inline.patch, 9758.txt
>
>
> nodetool compactionhistory may trigger NPE : 
> {code}
> admin@localhost:~$ nodetool compactionhistory
> Compaction History: 
> error: null
> -- StackTrace --
> java.lang.NullPointerException
>   at com.google.common.base.Joiner$MapJoiner.join(Joiner.java:330)
>   at org.apache.cassandra.utils.FBUtilities.toString(FBUtilities.java:515)
>   at 
> org.apache.cassandra.db.compaction.CompactionHistoryTabularData.from(CompactionHistoryTabularData.java:78)
>   

[jira] [Commented] (CASSANDRA-10286) Windows utest 3.0: org.apache.cassandra.io.sstable.SSTableLoaderTest.testLoadingIncompleteSSTable

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742762#comment-14742762
 ] 

Stefania commented on CASSANDRA-10286:
--

There is a unit test still failing on Windows: 
{{testWrongChecksumLastLineMissingFile}}. I've submitted a possible fix, CI is 
still pending.

> Windows utest 3.0: 
> org.apache.cassandra.io.sstable.SSTableLoaderTest.testLoadingIncompleteSSTable
> -
>
> Key: CASSANDRA-10286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10286
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Philip Thompson
>Assignee: Stefania
> Fix For: 3.x
>
>
> Distinct error message from CASSANDRA-10210.
> {code}
> junit.framework.AssertionFailedError: 
>   at 
> org.apache.cassandra.io.sstable.SSTableLoaderTest.testLoadingIncompleteSSTable(SSTableLoaderTest.java:186)
> {code}
> which I believe is from
> {code}
> ERROR 22:05:57 Unable to delete 
> D:\temp\1441663555277-0\SSTableLoaderTest\Standard2\ma-1-big-Data.db
> java.nio.file.AccessDeniedException: 
> D:\temp\1441663555277-0\SSTableLoaderTest\Standard2\ma-1-big-Data.db
>   at 
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83) 
> ~[na:1.8.0_51]
>   at 
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) 
> ~[na:1.8.0_51]
>   at 
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) 
> ~[na:1.8.0_51]
>   at 
> sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
>  ~[na:1.8.0_51]
>   at 
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
>  ~[na:1.8.0_51]
>   at java.nio.file.Files.delete(Files.java:1126) ~[na:1.8.0_51]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog.delete(TransactionLog.java:792)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog.access$100(TransactionLog.java:97)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionFile.lambda$deleteRecord$108(TransactionLog.java:504)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionFile$$Lambda$108/1799732569.accept(Unknown
>  Source) [main/:na]
>   at java.util.Arrays$ArrayList.forEach(Arrays.java:3880) [na:1.8.0_51]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionFile.deleteRecord(TransactionLog.java:504)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionFile$$Lambda$88/744169296.accept(Unknown
>  Source) [main/:na]
>   at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) 
> [na:1.8.0_51]
>   at 
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) 
> [na:1.8.0_51]
>   at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1540) 
> [na:1.8.0_51]
>   at 
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512) 
> [na:1.8.0_51]
>   at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502) 
> [na:1.8.0_51]
>   at 
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) 
> [na:1.8.0_51]
>   at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
>  [na:1.8.0_51]
>   at 
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) 
> [na:1.8.0_51]
>   at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) 
> [na:1.8.0_51]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionFile.deleteRecords(TransactionLog.java:490)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionData.removeUnfinishedLeftovers(TransactionLog.java:622)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionTidier.run(TransactionLog.java:837)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog$TransactionTidier.tidy(TransactionLog.java:822)
>  [main/:na]
>   at 
> org.apache.cassandra.utils.concurrent.Ref$GlobalState.release(Ref.java:294) 
> [main/:na]
>   at 
> org.apache.cassandra.utils.concurrent.Ref$State.ensureReleased(Ref.java:172) 
> [main/:na]
>   at 
> org.apache.cassandra.utils.concurrent.Ref.ensureReleased(Ref.java:92) 
> [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog.complete(TransactionLog.java:950)
>  [main/:na]
>   at 
> org.apache.cassandra.db.lifecycle.TransactionLog.doAbort(TransactionLog.java:969)
>  [main/:na]
>   at 
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:144)
>  [main/:na]
>   at 
> 

[jira] [Commented] (CASSANDRA-7392) Abort in-progress queries that time out

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742836#comment-14742836
 ] 

Stefania commented on CASSANDRA-7392:
-

I propose to log at WARN a generic message such as _some CQL queries timed out_ 
using the no spam logger to avoid polluting the main log file. Then, we log the 
full details at DEBUG level and follow what [~pauloricardomg] is doing for 
CASSANDRA-10241 by either adding a new appender writing to _query.log_ or by 
using _debug.log_. Does this sound OK?

bq. Is it just a few seconds? I would be ok with a handful of seconds.

It's 4-5 seconds on my box and 8-9 seconds on Jenkins.

bq. I think other slow query logs can include the full parameters of the query. 
Maybe we want to consider logging the fully formatted CQL statement or having 
that option. You can end up with a query that has a reasonable execution plan, 
but the bound parameters involved make it slow. Maybe we log the query up to a 
maximum size by default and people can increase it via a property.

At this level in {{ReadCommand}} the query is already bound. So the only 
parameter we could add is the maximum query size. I am not sure we should 
truncate the statement if we are logging at DEBUG level in a separate log file.

bq. For the properties, there is a constant for the "cassandra." in 
DatabaseDescriptor or Config (I can't recall)

{{Config.PROPERTY_PREFIX}}, thanks.

bq. I don't quite understand what min, max, and avg are telling me? I guess 
it's informative if I missed hitting the timeout at the expected time?

It's to give an indication on how timely the expiration thread is, as you 
suggested in the first round of the code review.

> Abort in-progress queries that time out
> ---
>
> Key: CASSANDRA-7392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7392
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 3.x
>
>
> Currently we drop queries that time out before we get to them (because node 
> is overloaded) but not queries that time out while being processed.  
> (Particularly common for index queries on data that shouldn't be indexed.)  
> Adding the latter and logging when we have to interrupt one gets us a poor 
> man's "slow query log" for free.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds

[Steven Lowenthal (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742914#comment-14742914
 ] 

Steven Lowenthal edited comment on CASSANDRA-8072 at 9/14/15 4:49 AM:
--

I can easily reproduce this with my automated launcher  I even tried to better 
randomize when non-seed nodes come in to join.   I recently noticed that the 
seed node has a socket stuck in CLOSE_WAIT for the nodes that report can't 
gossip with any seeds.  Perhaps the solution lies in ensuring that both ends of 
the connection properly close the connection.  It's likely the client (the node 
asking to join) exceptions out and dies without elegantly closing the 
connection.   See Screenshot.  Also well-known port afs3-fileserver is 7000.


was (Author: slowenthal):
I can easily reproduce this with my automated launcher  I even tried to better 
randomize when non-seed nodes come in to join.   I recently noticed that the 
seed node has a socket stuck in CLOSE_WAIT for the nodes that report can't 
gossip with any seeds.  Perhaps the solution lies in ensuring that both ends of 
the connection properly close the connection.  It's likely the client (the node 
asking to join) exceptions out and dies without elegantly closing the 
connection.   

> Exception during startup: Unable to gossip with any seeds
> -
>
> Key: CASSANDRA-8072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8072
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ryan Springer
>Assignee: Brandon Williams
> Fix For: 2.1.x
>
> Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, 
> cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2, 
> cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, 
> casandra-system-log-with-assert-patch.log, screenshot-1.png, 
> trace_logs.tar.bz2
>
>
> When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster 
> in either ec2 or locally, an error occurs sometimes with one of the nodes 
> refusing to start C*.  The error in the /var/log/cassandra/system.log is:
> ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) 
> Exception encountered during startup
> java.lang.RuntimeException: Unable to gossip with any seeds
> at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200)
> at 
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444)
> at 
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:609)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:502)
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java 
> (line 1279) Announcing shutdown
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 
> MessagingService.java (line 701) Waiting for messaging service to quiesce
>  INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 
> MessagingService.java (line 941) MessagingService has terminated the accept() 
> thread
> This errors does not always occur when provisioning a 2-node cluster, but 
> probably around half of the time on only one of the nodes.  I haven't been 
> able to reproduce this error with DSC 2.0.9, and there have been no code or 
> definition file changes in Opscenter.
> I can reproduce locally with the above steps.  I'm happy to test any proposed 
> fixes since I'm the only person able to reproduce reliably so far.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds

[Steven Lowenthal (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742914#comment-14742914
 ] 

Steven Lowenthal commented on CASSANDRA-8072:
-

I can easily reproduce this with my automated launcher  I even tried to better 
randomize when non-seed nodes come in to join.   I recently noticed that the 
seed node has a socket stuck in CLOSE_WAIT for the nodes that report can't 
gossip with any seeds.  Perhaps the solution lies in ensuring that both ends of 
the connection properly close the connection.  It's likely the client (the node 
asking to join) exceptions out and dies without elegantly closing the 
connection.   

> Exception during startup: Unable to gossip with any seeds
> -
>
> Key: CASSANDRA-8072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8072
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ryan Springer
>Assignee: Brandon Williams
> Fix For: 2.1.x
>
> Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, 
> cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2, 
> cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, 
> casandra-system-log-with-assert-patch.log, trace_logs.tar.bz2
>
>
> When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster 
> in either ec2 or locally, an error occurs sometimes with one of the nodes 
> refusing to start C*.  The error in the /var/log/cassandra/system.log is:
> ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) 
> Exception encountered during startup
> java.lang.RuntimeException: Unable to gossip with any seeds
> at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200)
> at 
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444)
> at 
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:609)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:502)
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java 
> (line 1279) Announcing shutdown
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 
> MessagingService.java (line 701) Waiting for messaging service to quiesce
>  INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 
> MessagingService.java (line 941) MessagingService has terminated the accept() 
> thread
> This errors does not always occur when provisioning a 2-node cluster, but 
> probably around half of the time on only one of the nodes.  I haven't been 
> able to reproduce this error with DSC 2.0.9, and there have been no code or 
> definition file changes in Opscenter.
> I can reproduce locally with the above steps.  I'm happy to test any proposed 
> fixes since I'm the only person able to reproduce reliably so far.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds

[Steven Lowenthal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Lowenthal updated CASSANDRA-8072:

Attachment: screenshot-1.png

> Exception during startup: Unable to gossip with any seeds
> -
>
> Key: CASSANDRA-8072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8072
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ryan Springer
>Assignee: Brandon Williams
> Fix For: 2.1.x
>
> Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, 
> cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2, 
> cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, 
> casandra-system-log-with-assert-patch.log, screenshot-1.png, 
> trace_logs.tar.bz2
>
>
> When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster 
> in either ec2 or locally, an error occurs sometimes with one of the nodes 
> refusing to start C*.  The error in the /var/log/cassandra/system.log is:
> ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) 
> Exception encountered during startup
> java.lang.RuntimeException: Unable to gossip with any seeds
> at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200)
> at 
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444)
> at 
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:609)
> at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:502)
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java 
> (line 1279) Announcing shutdown
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 
> MessagingService.java (line 701) Waiting for messaging service to quiesce
>  INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 
> MessagingService.java (line 941) MessagingService has terminated the accept() 
> thread
> This errors does not always occur when provisioning a 2-node cluster, but 
> probably around half of the time on only one of the nodes.  I haven't been 
> able to reproduce this error with DSC 2.0.9, and there have been no code or 
> definition file changes in Opscenter.
> I can reproduce locally with the above steps.  I'm happy to test any proposed 
> fixes since I'm the only person able to reproduce reliably so far.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)