[jira] [Commented] (CASSANDRA-11892) Can not replace a dead host

[Dikang Gu (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352206#comment-15352206
 ] 

Dikang Gu commented on CASSANDRA-11892:
---

[~brandon.williams] yeah, I also suspect the fix was probably not right. 
Anyway, I've assassinate the bad node. So why the host id would be missing?

> Can not replace a dead host
> ---
>
> Key: CASSANDRA-11892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11892
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dikang Gu
> Attachments: 0001-handle-hibernate-case.patch
>
>
> I got some errors when trying to replace a dead host.
> {code}
> 2016-05-25_20:59:37.61838 ERROR 20:59:37 [main]: Exception encountered during 
> startup
> 2016-05-25_20:59:37.61839 java.lang.UnsupportedOperationException: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:925)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:740)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:617)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:389) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
>  [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61841   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61910 Exception encountered during startup: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> {code}
> the status of the node is DN:
> {code}
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address  Load   Tokens  OwnsHost ID   
> Rack
> DN  2401:db00:2050:4196:face:0:13:0  809.83 GB  256 ?   null  
> ash5-04-pp
> {code}
> I add some logging and find something like this:
> {code}
> 2016-05-25_20:58:33.44305 INFO  20:58:33 [main]: Gathering node replacement 
> information for /2401:db00:2050:4196:face:0:13:0
> 2016-05-25_20:58:34.36966 INFO  20:58:34 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12167 INFO  20:58:41 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12248 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state STATUS
> 2016-05-25_20:58:41.12250 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 movename hibernate
> 2016-05-25_20:58:41.12252 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state LOAD
> {code}
> I find in the StorageService.onChange, we do not handle the "hibernate" 
> VersionValue, does it cause the problem?
> Is it safe to apply the patch to fix it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11393) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352183#comment-15352183
 ] 

Stefania commented on CASSANDRA-11393:
--

If we set it by default, it might slow things things a bit, you need to perhaps 
measure the time difference in running the tests before deciding. This setting 
will show us the call stack of LEAKED resources, therefore allowing us to fix 
the LEAK errors reported by CASSANDRA-11767.

> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11393
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Streaming and Messaging
>Reporter: Philip Thompson
>Assignee: Benjamin Lerer
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: 11393-3.0.txt
>
>
> We are seeing a failure in the upgrade tests that go from 2.1 to 3.0
> {code}
> node2: ERROR [SharedPool-Worker-2] 2016-03-10 20:05:17,865 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xeb79b477, 
> /127.0.0.1:39613 => /127.0.0.2:9042]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1208)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1155)
>  ~[main/:na]
>   at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor$AlwaysSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:330)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.SinglePartitionReadCommand.execute(SinglePartitionReadCommand.java:302)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:67)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.SinglePartitionPager.fetchPage(SinglePartitionPager.java:34)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:297)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:333)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:209)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:472)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:449)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.Abst

[2/2] cassandra git commit: update to version 3.10

[jake]

update to version 3.10


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/01343483
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/01343483
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/01343483

Branch: refs/heads/trunk
Commit: 01343483316d895051d23b29032e1d2ecf35aab9
Parents: 7d4a71f
Author: T Jake Luciani 
Authored: Mon Jun 27 21:02:37 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 27 21:02:37 2016 -0400

--
 build.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/01343483/build.xml
--
diff --git a/build.xml b/build.xml
index 5ddf40d..85596de 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

[1/2] cassandra git commit: update to version 3.9

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/trunk d9a4c78a3 -> 013434833


update to version 3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d4a71f6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d4a71f6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d4a71f6

Branch: refs/heads/trunk
Commit: 7d4a71f63617ffcffcadd8b0b35715770d361fc9
Parents: d9a4c78
Author: T Jake Luciani 
Authored: Mon Jun 27 21:01:32 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 27 21:01:32 2016 -0400

--
 build.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d4a71f6/build.xml
--
diff --git a/build.xml b/build.xml
index af33f99..5ddf40d 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

cassandra git commit: update to version 3.9

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.9 [created] 7d4a71f63


update to version 3.9


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d4a71f6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d4a71f6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d4a71f6

Branch: refs/heads/cassandra-3.9
Commit: 7d4a71f63617ffcffcadd8b0b35715770d361fc9
Parents: d9a4c78
Author: T Jake Luciani 
Authored: Mon Jun 27 21:01:32 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 27 21:01:32 2016 -0400

--
 build.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d4a71f6/build.xml
--
diff --git a/build.xml b/build.xml
index af33f99..5ddf40d 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 
 
 
-
+
 
 
 http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/>

[cassandra] Git Push Summary

[jake]

Repository: cassandra
Updated Tags:  refs/tags/3.8-tentative [created] d9a4c78a3

cassandra git commit: bump deb version

[jake]

Repository: cassandra
Updated Branches:
  refs/heads/trunk c7b9401bf -> d9a4c78a3


bump deb version


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d9a4c78a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d9a4c78a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d9a4c78a

Branch: refs/heads/trunk
Commit: d9a4c78a32d893cb2e05fed0e9c5a2e58001b567
Parents: c7b9401
Author: T Jake Luciani 
Authored: Mon Jun 27 20:42:01 2016 -0400
Committer: T Jake Luciani 
Committed: Mon Jun 27 20:42:01 2016 -0400

--
 debian/changelog | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d9a4c78a/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 44db441..e071140 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (3.8) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani   Mon, 27 Jun 2016 20:40:40 -0400
+
 cassandra (3.6) unstable; urgency=medium
 
   * New release

[jira] [Commented] (CASSANDRA-11393) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352106#comment-15352106
 ] 

Russ Hatch commented on CASSANDRA-11393:


dtest branch here: 
https://github.com/riptano/cassandra-dtest/compare/ca11393?expand=1
upgrade job queued here: 
http://cassci.datastax.com/view/Upgrades/job/upgrade_tests-all-custom_branch_runs/35/

don't panic if the job above fails, I've been having some intermittent problems 
with that job failing lately, can re-run if needed.

> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11393
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Streaming and Messaging
>Reporter: Philip Thompson
>Assignee: Benjamin Lerer
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: 11393-3.0.txt
>
>
> We are seeing a failure in the upgrade tests that go from 2.1 to 3.0
> {code}
> node2: ERROR [SharedPool-Worker-2] 2016-03-10 20:05:17,865 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xeb79b477, 
> /127.0.0.1:39613 => /127.0.0.2:9042]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1208)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1155)
>  ~[main/:na]
>   at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor$AlwaysSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:330)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.SinglePartitionReadCommand.execute(SinglePartitionReadCommand.java:302)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:67)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.SinglePartitionPager.fetchPage(SinglePartitionPager.java:34)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:297)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:333)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:209)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:472)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:449)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.

[jira] [Commented] (CASSANDRA-12073) [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial results

[DOAN DuyHai (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352099#comment-15352099
 ] 

DOAN DuyHai commented on CASSANDRA-12073:
-

OK, I'll update the patch with your remarks [~xedin]

> [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial 
> results
> ---
>
> Key: CASSANDRA-12073
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12073
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.7
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
> Fix For: 3.x
>
> Attachments: patch_PREFIX_search_with_CONTAINS_mode.txt
>
>
> {noformat}
> cqlsh:music> CREATE TABLE music.albums (
> id uuid PRIMARY KEY,
> artist text,
> country text,
> quality text,
> status text,
> title text,
> year int
> );
> cqlsh:music> CREATE CUSTOM INDEX albums_artist_idx ON music.albums (artist) 
> USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 
> 'CONTAINS', 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> cqlsh:music> SELECT * FROM albums WHERE artist like 'lady%'  LIMIT 100;
>  id   | artist| country| quality 
> | status| title | year
> --+---++-+---+---+--
>  372bb0ab-3263-41bc-baad-bb520ddfa787 | Lady Gaga |USA |  normal 
> |  Official |   Red and Blue EP | 2006
>  1a4abbcd-b5de-4c69-a578-31231e01ff09 | Lady Gaga |Unknown |  normal 
> | Promotion |Poker Face | 2008
>  31f4a0dc-9efc-48bf-9f5e-bfc09af42b82 | Lady Gaga |USA |  normal 
> |  Official |   The Cherrytree Sessions | 2009
>  8ebfaebd-28d0-477d-b735-469661ce6873 | Lady Gaga |Unknown |  normal 
> |  Official |Poker Face | 2009
>  98107d82-e0dd-46bc-a273-1577578984c7 | Lady Gaga |USA |  normal 
> |  Official |   Just Dance: The Remixes | 2008
>  a76af0f2-f5c5-4306-974a-e3c17158e6c6 | Lady Gaga |  Italy |  normal 
> |  Official |  The Fame | 2008
>  849ee019-8b15-4767-8660-537ab9710459 | Lady Gaga |USA |  normal 
> |  Official |Christmas Tree | 2008
>  4bad59ac-913f-43da-9d48-89adc65453d2 | Lady Gaga |  Australia |  normal 
> |  Official | Eh Eh | 2009
>  80327731-c450-457f-bc12-0a8c21fd9c5d | Lady Gaga |USA |  normal 
> |  Official | Just Dance Remixes Part 2 | 2008
>  3ad33659-e932-4d31-a040-acab0e23c3d4 | Lady Gaga |Unknown |  normal 
> |  null |Just Dance | 2008
>  9adce7f6-6a1d-49fd-b8bd-8f6fac73558b | Lady Gaga | United Kingdom |  normal 
> |  Official |Just Dance | 2009
> (11 rows)
> {noformat}
> *SASI* says that there are only 11 artists whose name starts with {{lady}}.
> However, in the data set, there are:
> * Lady Pank
> * Lady Saw
> * Lady Saw
> * Ladyhawke
> * Ladytron
> * Ladysmith Black Mambazo
> * Lady Gaga
> * Lady Sovereign
> etc ...
> By debugging the source code, the issue is in 
> {{OnDiskIndex.TermIterator::computeNext()}}
> {code:java}
> for (;;)
> {
> if (currentBlock == null)
> return endOfData();
> if (offset >= 0 && offset < currentBlock.termCount())
> {
> DataTerm currentTerm = currentBlock.getTerm(nextOffset());
> if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
> continue;
> // flip the flag right on the first bounds match
> // to avoid expensive comparisons
> checkLower = false;
> if (checkUpper && !e.isUpperSatisfiedBy(currentTerm))
> return endOfData();
> return currentTerm;
> }
> nextBlock();
> }
> {code}
>  So the {{endOfData()}} conditions are:
> * currentBlock == null
> * checkUpper && !e.isUpperSatisfiedBy(currentTerm)
> The problem is that {{e::isUpperSatisfiedBy}} is checking not only whether 
> the term match but also returns *false* when it's a *partial term* !
> {code:java}
> public boolean isUpperSatisfiedBy(OnDiskIndex.DataTerm term)
> {
> if (!hasUpper())
> return true;
> if (nonMatchingPartial(term))
> return false;
> int cmp = term.compareTo(validator, upper.value, false);
> return cmp < 0 || cmp == 0 && upper.inclusive;
> }
> {code}
> By debugging the OnDiskIndex data, I've found:

[jira] [Commented] (CASSANDRA-12073) [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial results

[Pavel Yaskevich (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352073#comment-15352073
 ] 

Pavel Yaskevich commented on CASSANDRA-12073:
-

[~doanduyhai] The changes look good to me, because in case of prefix queries we 
should "step over" partial terms (because they are essentially invisible for 
the expression in PREFIX mode) until upper bound tells us to stop instead of 
trying to stop right when partial has been encountered. 

Couple of comments regarding code: since partialTermAndPrefixOperation is only 
used at one place I would rather have it as inline if with the comment on top 
e.g. 
{noformat}
// we need to step over all of the partial terms, in PREFIX mode,
// encountered by the query until upper-bound tells us to stop
if (operation == Op.PREFIX && term.isPartial()) 
   continue;

// haven't reached the start of the query range yet, let's 
// keep skip the current term until lower bound is satisfied
if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
   continue;
{noformat}

Also I think it essential to have a test-case in OnDiskIndexTest since it gives 
you more control over index blocks and internals than SASIndexTest.

> [SASI] PREFIX search on CONTAINS/NonTokenizer mode returns only partial 
> results
> ---
>
> Key: CASSANDRA-12073
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12073
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.7
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
> Fix For: 3.x
>
> Attachments: patch_PREFIX_search_with_CONTAINS_mode.txt
>
>
> {noformat}
> cqlsh:music> CREATE TABLE music.albums (
> id uuid PRIMARY KEY,
> artist text,
> country text,
> quality text,
> status text,
> title text,
> year int
> );
> cqlsh:music> CREATE CUSTOM INDEX albums_artist_idx ON music.albums (artist) 
> USING 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 
> 'CONTAINS', 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> cqlsh:music> SELECT * FROM albums WHERE artist like 'lady%'  LIMIT 100;
>  id   | artist| country| quality 
> | status| title | year
> --+---++-+---+---+--
>  372bb0ab-3263-41bc-baad-bb520ddfa787 | Lady Gaga |USA |  normal 
> |  Official |   Red and Blue EP | 2006
>  1a4abbcd-b5de-4c69-a578-31231e01ff09 | Lady Gaga |Unknown |  normal 
> | Promotion |Poker Face | 2008
>  31f4a0dc-9efc-48bf-9f5e-bfc09af42b82 | Lady Gaga |USA |  normal 
> |  Official |   The Cherrytree Sessions | 2009
>  8ebfaebd-28d0-477d-b735-469661ce6873 | Lady Gaga |Unknown |  normal 
> |  Official |Poker Face | 2009
>  98107d82-e0dd-46bc-a273-1577578984c7 | Lady Gaga |USA |  normal 
> |  Official |   Just Dance: The Remixes | 2008
>  a76af0f2-f5c5-4306-974a-e3c17158e6c6 | Lady Gaga |  Italy |  normal 
> |  Official |  The Fame | 2008
>  849ee019-8b15-4767-8660-537ab9710459 | Lady Gaga |USA |  normal 
> |  Official |Christmas Tree | 2008
>  4bad59ac-913f-43da-9d48-89adc65453d2 | Lady Gaga |  Australia |  normal 
> |  Official | Eh Eh | 2009
>  80327731-c450-457f-bc12-0a8c21fd9c5d | Lady Gaga |USA |  normal 
> |  Official | Just Dance Remixes Part 2 | 2008
>  3ad33659-e932-4d31-a040-acab0e23c3d4 | Lady Gaga |Unknown |  normal 
> |  null |Just Dance | 2008
>  9adce7f6-6a1d-49fd-b8bd-8f6fac73558b | Lady Gaga | United Kingdom |  normal 
> |  Official |Just Dance | 2009
> (11 rows)
> {noformat}
> *SASI* says that there are only 11 artists whose name starts with {{lady}}.
> However, in the data set, there are:
> * Lady Pank
> * Lady Saw
> * Lady Saw
> * Ladyhawke
> * Ladytron
> * Ladysmith Black Mambazo
> * Lady Gaga
> * Lady Sovereign
> etc ...
> By debugging the source code, the issue is in 
> {{OnDiskIndex.TermIterator::computeNext()}}
> {code:java}
> for (;;)
> {
> if (currentBlock == null)
> return endOfData();
> if (offset >= 0 && offset < currentBlock.termCount())
> {
> DataTerm currentTerm = currentBlock.getTerm(nextOffset());
> if (checkLower && !e.isLowerSatisfiedBy(currentTerm))
> continue;
> // flip the flag right on the first bounds match
> // to avoid expensi

[jira] [Commented] (CASSANDRA-12043) Syncing most recent commit in CAS across replicas can cause all CAS queries in the CQL partition to fail

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352028#comment-15352028
 ] 

sankalp kohli commented on CASSANDRA-12043:
---

[~slebresne] Please commit it. Thanks 

> Syncing most recent commit in CAS across replicas can cause all CAS queries 
> in the CQL partition to fail
> 
>
> Key: CASSANDRA-12043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12043
> Project: Cassandra
>  Issue Type: Bug
>Reporter: sankalp kohli
>Assignee: Sylvain Lebresne
>
> We update the most recent commit on requiredParticipant replicas if out of 
> sync during the prepare round in beginAndRepairPaxos method. We keep doing 
> this in a loop till the requiredParticipant replicas have the same most 
> recent commit or we hit timeout. 
> Say we have 3 machines A,B and C and gc grace on the table is 10 days. We do 
> a CAS write at time 0 and it went to A and B but not to C.  C will get the 
> hint later but will not update the most recent commit in paxos table. This is 
> how CAS hints work. 
> In the paxos table whose gc_grace=0, most_recent_commit in A and B will be 
> inserted with timestamp 0 and with a TTL of 10 days. After 10 days, this 
> insert will become a tombstone at time 0 till it is compacted away since 
> gc_grace=0.
> Do a CAS read after say 1 day on the same CQL partition and this time prepare 
> phase involved A and C. most_recent_commit on C for this CQL partition is 
> empty. A sends the most_recent_commit to C with a timestamp of 0 and with a 
> TTL of 10 days. This most_recent_commit on C will expire on 11th day since it 
> is inserted after 1 day. 
> most_recent_commit are now in sync on A,B and C, however A and B 
> most_recent_commit will expire on 10th day whereas for C it will expire on 
> 11th day since it was inserted one day later. 
> Do another CAS read after 10days when most_recent_commit on A and B have 
> expired and is treated as tombstones till compacted. In this CAS read, say A 
> and C are involved in prepare phase. most_recent_commit will not match 
> between them since it is expired in A and is still there on C. This will 
> cause most_recent_commit to be applied to A with a timestamp of 0 and TTL of 
> 10 days. If A has not compacted away the original most_recent_commit which 
> has expired, this new write to most_recent_commit wont be visible on reads 
> since there is a tombstone with same timestamp(Delete wins over data with 
> same timestamp). 
> Another round of prepare will follow and again A would say it does not know 
> about most_recent_write(covered by original write which is not a tombstone) 
> and C will again try to send the write to A. This can keep going on till the 
> request timeouts or only A and B are involved in the prepare phase. 
> When A’s original most_recent_commit which is now a tombstone is compacted, 
> all the inserts which it was covering will come live. This will in turn again 
> get played to another replica. This ping pong can keep going on for a long 
> time. 
> The issue is that most_recent_commit is expiring at different times across 
> replicas. When they get replayed to a replica to make it in sync, we again 
> set the TTL from that point.  
> During the CAS read which timed out, most_recent_commit was being sent to 
> another replica in a loop. Even in successful requests, it will try to loop 
> for a couple of times if involving A and C and then when the replicas which 
> respond are A and B, it will succeed. So this will have impact on latencies 
> as well. 
> These timeouts gets worse when a machine is down as no progress can be made 
> as the machine with unexpired commit is always involved in the CAS prepare 
> round. Also with range movements, the new machine gaining range has empty 
> most recent commit and gets the commit at a later time causing same issue. 
> Repro steps:
> 1. Paxos TTL is max(3 hours, gc_grace) as defined in 
> SystemKeyspace.paxosTtl(). Change this method to not put a minimum TTL of 3 
> hours. 
> Method  SystemKeyspace.paxosTtl() will look like return 
> metadata.getGcGraceSeconds();   instead of return Math.max(3 * 3600, 
> metadata.getGcGraceSeconds());
> We are doing this so that we dont need to wait for 3 hours. 
> Create a 3 node cluster with the code change suggested above with machines 
> A,B and C
> CREATE KEYSPACE  test WITH REPLICATION = { 'class' : 'SimpleStrategy', 
> 'replication_factor' : 3 };
> use test;
> CREATE TABLE users (a int PRIMARY KEY,b int);
> alter table users WITH gc_grace_seconds=120;
> consistency QUORUM;
> bring down machine C
> INSERT INTO users (user_name, password ) VALUES ( 1,1) IF NOT EXISTS;
> Nodetool flush on machine A and B
> Bring up the down machine B 
> consis

[jira] [Commented] (CASSANDRA-12034) Special handling for Netty's direct memory allocation failure

[T Jake Luciani (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351821#comment-15351821
 ] 

T Jake Luciani commented on CASSANDRA-12034:


+1

> Special handling for Netty's direct memory allocation failure
> -
>
> Key: CASSANDRA-12034
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12034
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>  Labels: doc-impacting
> Fix For: 3.x
>
>
> With CASSANDRA-12032, Netty throws a 
> {{io.netty.util.internal.OutOfDirectMemoryError}} if there's not enough 
> off-heap memory for the response buffer. We can easily handle this situation 
> and return an error. This is not a condition that destabilizes the system and 
> should therefore not passed to {{JVMStabilityInspector}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-12081) CommitLogStressTest failing post-CASSANDRA-8844

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie resolved CASSANDRA-12081.
-
   Resolution: Duplicate
 Reviewer:   (was: Carl Yeksigian)
Fix Version/s: (was: 3.x)

> CommitLogStressTest failing post-CASSANDRA-8844
> ---
>
> Key: CASSANDRA-12081
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12081
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
> Attachments: 0001-Fix-CommitLogStressTest.patch
>
>
> Test timing out after CASSANDRA-8844.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8876) Allow easier init script reuse

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351718#comment-15351718
 ] 

Michael Shuler commented on CASSANDRA-8876:
---

Thanks for the patch. I'll test out how this works for default installs and see 
how it goes.

> Allow easier init script reuse
> --
>
> Key: CASSANDRA-8876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8876
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Marko Asplund
>Assignee: Michael Shuler
> Fix For: 3.x
>
> Attachments: trunk-CASSANDRA-8876.txt
>
>
> Make it possible to reuse the Cassandra debian init script with different 
> configuration and Cassandra home paths by making paths configurable via 
> environment variables set in /etc/default/cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8876) Allow easier init script reuse

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler reassigned CASSANDRA-8876:
-

Assignee: Michael Shuler

> Allow easier init script reuse
> --
>
> Key: CASSANDRA-8876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8876
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Marko Asplund
>Assignee: Michael Shuler
> Fix For: 3.x
>
> Attachments: trunk-CASSANDRA-8876.txt
>
>
> Make it possible to reuse the Cassandra debian init script with different 
> configuration and Cassandra home paths by making paths configurable via 
> environment variables set in /etc/default/cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11892) Can not replace a dead host

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351670#comment-15351670
 ] 

Brandon Williams commented on CASSANDRA-11892:
--

Your real problem seems to be a missing host_id for the down node.  You'll 
probably need to assassinate it.

> Can not replace a dead host
> ---
>
> Key: CASSANDRA-11892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11892
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dikang Gu
> Attachments: 0001-handle-hibernate-case.patch
>
>
> I got some errors when trying to replace a dead host.
> {code}
> 2016-05-25_20:59:37.61838 ERROR 20:59:37 [main]: Exception encountered during 
> startup
> 2016-05-25_20:59:37.61839 java.lang.UnsupportedOperationException: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:925)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:740)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:617)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:389) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
>  [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61841   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61910 Exception encountered during startup: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> {code}
> the status of the node is DN:
> {code}
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address  Load   Tokens  OwnsHost ID   
> Rack
> DN  2401:db00:2050:4196:face:0:13:0  809.83 GB  256 ?   null  
> ash5-04-pp
> {code}
> I add some logging and find something like this:
> {code}
> 2016-05-25_20:58:33.44305 INFO  20:58:33 [main]: Gathering node replacement 
> information for /2401:db00:2050:4196:face:0:13:0
> 2016-05-25_20:58:34.36966 INFO  20:58:34 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12167 INFO  20:58:41 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12248 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state STATUS
> 2016-05-25_20:58:41.12250 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 movename hibernate
> 2016-05-25_20:58:41.12252 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state LOAD
> {code}
> I find in the StorageService.onChange, we do not handle the "hibernate" 
> VersionValue, does it cause the problem?
> Is it safe to apply the patch to fix it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8076) Expose an mbean method to poll for repair job status

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8076:
--
Status: Open  (was: Patch Available)

> Expose an mbean method to poll for repair job status
> 
>
> Key: CASSANDRA-8076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8076
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Philip S Doctor
>Assignee: Yuki Morishita
> Attachments: 8076-2.0.txt
>
>
> Given the int reply-id from forceRepairAsync, allow a client to request the 
> status of this ID via jmx.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11892) Can not replace a dead host

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-11892:
-
Status: Open  (was: Patch Available)

> Can not replace a dead host
> ---
>
> Key: CASSANDRA-11892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11892
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dikang Gu
> Attachments: 0001-handle-hibernate-case.patch
>
>
> I got some errors when trying to replace a dead host.
> {code}
> 2016-05-25_20:59:37.61838 ERROR 20:59:37 [main]: Exception encountered during 
> startup
> 2016-05-25_20:59:37.61839 java.lang.UnsupportedOperationException: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:925)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:740)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:617)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:389) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
>  [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61841   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61910 Exception encountered during startup: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> {code}
> the status of the node is DN:
> {code}
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address  Load   Tokens  OwnsHost ID   
> Rack
> DN  2401:db00:2050:4196:face:0:13:0  809.83 GB  256 ?   null  
> ash5-04-pp
> {code}
> I add some logging and find something like this:
> {code}
> 2016-05-25_20:58:33.44305 INFO  20:58:33 [main]: Gathering node replacement 
> information for /2401:db00:2050:4196:face:0:13:0
> 2016-05-25_20:58:34.36966 INFO  20:58:34 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12167 INFO  20:58:41 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12248 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state STATUS
> 2016-05-25_20:58:41.12250 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 movename hibernate
> 2016-05-25_20:58:41.12252 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state LOAD
> {code}
> I find in the StorageService.onChange, we do not handle the "hibernate" 
> VersionValue, does it cause the problem?
> Is it safe to apply the patch to fix it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11892) Can not replace a dead host

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351669#comment-15351669
 ] 

Brandon Williams commented on CASSANDRA-11892:
--

Whatever has happened here, this patch is not the correct fix.  We don't handle 
hibernate on purpose, since a replacing node shouldn't generate any events, 
making it invisible while it is doing the replace.

> Can not replace a dead host
> ---
>
> Key: CASSANDRA-11892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11892
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dikang Gu
> Attachments: 0001-handle-hibernate-case.patch
>
>
> I got some errors when trying to replace a dead host.
> {code}
> 2016-05-25_20:59:37.61838 ERROR 20:59:37 [main]: Exception encountered during 
> startup
> 2016-05-25_20:59:37.61839 java.lang.UnsupportedOperationException: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:925)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:740)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61839   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:617)
>  ~[apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:389) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61840   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
>  [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61841   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) 
> [apache-cassandra-2.1.14+git20160523.7442267.jar:2.1.14+git20160523.7442267]
> 2016-05-25_20:59:37.61910 Exception encountered during startup: Cannot 
> replace token 100284002935427428580945058996711341062 which does not exist!
> {code}
> the status of the node is DN:
> {code}
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address  Load   Tokens  OwnsHost ID   
> Rack
> DN  2401:db00:2050:4196:face:0:13:0  809.83 GB  256 ?   null  
> ash5-04-pp
> {code}
> I add some logging and find something like this:
> {code}
> 2016-05-25_20:58:33.44305 INFO  20:58:33 [main]: Gathering node replacement 
> information for /2401:db00:2050:4196:face:0:13:0
> 2016-05-25_20:58:34.36966 INFO  20:58:34 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12167 INFO  20:58:41 [GossipStage:1]: InetAddress 
> /2401:db00:2050:4196:face:0:13:0 is now DOWN
> 2016-05-25_20:58:41.12248 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state STATUS
> 2016-05-25_20:58:41.12250 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 movename hibernate
> 2016-05-25_20:58:41.12252 INFO  20:58:41 [GossipStage:1]: Node 
> /2401:db00:2050:4196:face:0:13:0 state LOAD
> {code}
> I find in the StorageService.onChange, we do not handle the "hibernate" 
> VersionValue, does it cause the problem?
> Is it safe to apply the patch to fix it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10965) Shadowable tombstones can continue to shadow view results when timestamps match

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-10965:
---
Status: Open  (was: Patch Available)

> Shadowable tombstones can continue to shadow view results when timestamps 
> match
> ---
>
> Key: CASSANDRA-10965
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10965
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
> Fix For: 3.0.x
>
> Attachments: shadow-ts.cql
>
>
> I've attached a script which reproduces the issue. The first time we insert 
> with {{TIMESTAMP 2}}, we are inserting a new row which has the same timestamp 
> as the previous shadow tombstone, and it continues to be shadowed by that 
> tombstone because we shadow values with the same timestamp.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8720) Provide tools for finding wide row/partition keys

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8720:
--
Status: Open  (was: Patch Available)

> Provide tools for finding wide row/partition keys
> -
>
> Key: CASSANDRA-8720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8720
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: J.B. Langston
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 8720.txt
>
>
> Multiple users have requested some sort of tool to help identify wide row 
> keys. They get into a situation where they know a wide row/partition has been 
> inserted and it's causing problems for them but they have no idea what the 
> row key is in order to remove it.  
> Maintaining the widest row key currently encountered and displaying it in 
> cfstats would be one possible approach.
> Another would be an offline tool (possibly an enhancement to sstablekeys) to 
> show the number of columns/bytes per key in each sstable. If a tool to 
> aggregate the information at a CF-level could be provided that would be a 
> bonus, but it shouldn't be too hard to write a script wrapper to aggregate 
> them if not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11550) Make the fanout size for LeveledCompactionStrategy to be configurable

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11550:
---
Status: Open  (was: Patch Available)

> Make the fanout size for LeveledCompactionStrategy to be configurable
> -
>
> Key: CASSANDRA-11550
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11550
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Compaction
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 3.x
>
> Attachments: 
> 0001-make-fanout-size-for-leveledcompactionstrategy-to-be.patch
>
>
> Currently, the fanout size for LeveledCompactionStrategy is hard coded in the 
> system (10). It would be useful to make the fanout size to be tunable, so 
> that we can change it according to different use cases.
> Further more, we can change the size dynamically.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11477) MerkleTree mismatch when a cell is shadowed by a range tombstone in different SSTables

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11477:
---
Status: Open  (was: Patch Available)

> MerkleTree mismatch when a cell is shadowed by a range tombstone in different 
> SSTables
> --
>
> Key: CASSANDRA-11477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11477
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Fabien Rousseau
>  Labels: repair
> Attachments: 11477-2.1.patch
>
>
> Below is a script which allows to reproduce the problem:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy', 
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 text,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3) VALUES ( 'a', 'b', 'c');
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush 
> ccm node1 cqlsh
> USE test_rt;
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair test_rt table1
> # now grep the log and observe that there was some inconstencies detected 
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> The wrong hash will be computed on node1, which will include the previously 
> deleted cell, thus resulting in a MT mismatch.
> This is due to the fact that, in LazilyCompactedRow, the RT is not added into 
> the RT tracker (in fact, it's added only if it is GCable, but should always 
> be added).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11706) Tracing payload not passed through newSession(..)

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351654#comment-15351654
 ] 

Jonathan Ellis commented on CASSANDRA-11706:


Is this still on your radar, [~mck]?

> Tracing payload not passed through newSession(..)
> -
>
> Key: CASSANDRA-11706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11706
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
>Priority: Minor
>  Labels: tracing
> Fix For: 3.x
>
> Attachments: 11706-trunk.txt
>
>
> Caused by CASSANDRA-10392
> There's a small bug in 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tracing/Tracing.java#L153
> {noformat}
> public UUID newSession(UUID sessionId, Map 
> customPayload)
> {
> return newSession(sessionId, TraceType.QUERY, Collections.EMPTY_MAP);
> }{noformat}
> in that it passes on an {{EMPTY_MAP}} instead of the {{customPayload}}.
> I've marked this as "minor" as custom tracing plugins can easily enough 
> workaround it by also overriding the {{newSession(UUID sessionId, 
> Map customPayload)}} method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11706) Tracing payload not passed through newSession(..)

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11706:
---
Status: Open  (was: Patch Available)

> Tracing payload not passed through newSession(..)
> -
>
> Key: CASSANDRA-11706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11706
> Project: Cassandra
>  Issue Type: Bug
>Reporter: mck
>Assignee: mck
>Priority: Minor
>  Labels: tracing
> Fix For: 3.x
>
> Attachments: 11706-trunk.txt
>
>
> Caused by CASSANDRA-10392
> There's a small bug in 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tracing/Tracing.java#L153
> {noformat}
> public UUID newSession(UUID sessionId, Map 
> customPayload)
> {
> return newSession(sessionId, TraceType.QUERY, Collections.EMPTY_MAP);
> }{noformat}
> in that it passes on an {{EMPTY_MAP}} instead of the {{customPayload}}.
> I've marked this as "minor" as custom tracing plugins can easily enough 
> workaround it by also overriding the {{newSession(UUID sessionId, 
> Map customPayload)}} method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8385) Clean up generics in uses of AbstractType

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8385:
--
Status: Open  (was: Patch Available)

> Clean up generics in uses of AbstractType
> -
>
> Key: CASSANDRA-8385
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8385
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local Write-Read Paths
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Trivial
> Attachments: 8385.patch
>
>
> Almost all uses of AbstractType are from code that doesn't know or care what 
> the specific type is and would be better served by a non-generic version of 
> the concept.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11519) Add support for IBM POWER

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11519:
---
Status: Open  (was: Patch Available)

> Add support for IBM POWER
> -
>
> Key: CASSANDRA-11519
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11519
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: POWER architecture
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: 11519-2.1.txt, 11519-3.0.txt
>
>
> Add support for the IBM POWER architecture (ppc, ppc64, and ppc64le) in 
> org.apache.cassandra.utils.FastByteOperations, 
> org.apache.cassandra.utils.memory.MemoryUtil, and 
> org.apache.cassandra.io.util.Memory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-12041) Add CDC to describe table

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351600#comment-15351600
 ] 

Adam Holmberg edited comment on CASSANDRA-12041 at 6/27/16 6:51 PM:


This update is in version 3.5.0 of the driver (released today).

I can provide a "patch" if needed, but it's really just a matter of updating 
the packaged driver.


was (Author: aholmber):
This update is in version 3.5.0 of the driver (released today).

> Add CDC to describe table
> -
>
> Key: CASSANDRA-12041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12041
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Carl Yeksigian
>Assignee: Joshua McKenzie
>  Labels: client-impacting
> Fix For: 3.x
>
>
> Currently we do not output CDC with {{DESCRIBE TABLE}}, but should include 
> that for 3.8+ tables.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-12041) Add CDC to describe table

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351600#comment-15351600
 ] 

Adam Holmberg commented on CASSANDRA-12041:
---

This update is in version 3.5.0 of the driver (released today).

> Add CDC to describe table
> -
>
> Key: CASSANDRA-12041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12041
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Carl Yeksigian
>Assignee: Joshua McKenzie
>  Labels: client-impacting
> Fix For: 3.x
>
>
> Currently we do not output CDC with {{DESCRIBE TABLE}}, but should include 
> that for 3.8+ tables.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351581#comment-15351581
 ] 

Sylvain Lebresne commented on CASSANDRA-8700:
-

As we wanted to have the new doc (even if incomplete) for 3.8 and we're about 
to freeze 3.8, I committed the branch. I'm sure everyone will agree this is 
better than what we have. Not that this is just so that some doc is included 
with the 3.8 artifacts, but I'll only push the doc online next week (with the 
release of 3.8) and we can still add/improve things till then.
So I suggest keeping this ticket open until next week, and if someone has 
updates to the doc he want to suggest, he can just put it here and I'll include 
it directly. Once 3.8 is out, I'll close this ticket and we can start using new 
ticket for new contributions normally.

> replace the wiki with docs in the git repo
> --
>
> Key: CASSANDRA-8700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8700
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.8
>
> Attachments: TombstonesAndGcGrace.md, bloom_filters.md, 
> compression.md, contributing.zip, getting_started.zip, hardware.md
>
>
> The wiki as it stands is pretty terrible.  It takes several minutes to apply 
> a single update, and as a result, it's almost never updated.  The information 
> there has very little context as to what version it applies to.  Most people 
> I've talked to that try to use the information they find there find it is 
> more confusing than helpful.
> I'd like to propose that instead of using the wiki, the doc directory in the 
> cassandra repo be used for docs (already used for CQL3 spec) in a format that 
> can be built to a variety of output formats like HTML / epub / etc.  I won't 
> start the bikeshedding on which markup format is preferable - but there are 
> several options that can work perfectly fine.  I've personally use sphinx w/ 
> restructured text, and markdown.  Both can build easily and as an added bonus 
> be pushed to readthedocs (or something similar) automatically.  For an 
> example, see cqlengine's documentation, which I think is already 
> significantly better than the wiki: 
> http://cqlengine.readthedocs.org/en/latest/
> In addition to being overall easier to maintain, putting the documentation in 
> the git repo adds context, since it evolves with the versions of Cassandra.
> If the wiki were kept even remotely up to date, I wouldn't bother with this, 
> but not having at least some basic documentation in the repo, or anywhere 
> associated with the project, is frustrating.
> For reference, the last 3 updates were:
> 1/15/15 - updating committers list
> 1/08/15 - updating contributers and how to contribute
> 12/16/14 - added a link to CQL docs from wiki frontpage (by me)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[21/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/cql/definitions.rst
--
diff --git a/doc/source/cql/definitions.rst b/doc/source/cql/definitions.rst
new file mode 100644
index 000..61ed47c
--- /dev/null
+++ b/doc/source/cql/definitions.rst
@@ -0,0 +1,225 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+Definitions
+---
+
+.. _conventions:
+
+Conventions
+^^^
+
+To aid in specifying the CQL syntax, we will use the following conventions in 
this document:
+
+- Language rules will be given in an informal `BNF variant
+  `_ notation. 
In particular, we'll use square brakets
+  (``[ item ]``) for optional items, ``*`` and ``+`` for repeated items (where 
``+`` imply at least one).
+- The grammar will also use the following convention for convenience: 
non-terminal term will be lowercase (and link to
+  their definition) while terminal keywords will be provided "all caps". Note 
however that keywords are
+  :ref:`identifiers` and are thus case insensitive in practice. We will also 
define some early construction using
+  regexp, which we'll indicate with ``re()``.
+- The grammar is provided for documentation purposes and leave some minor 
details out.  For instance, the comma on the
+  last column definition in a ``CREATE TABLE`` statement is optional but 
supported if present even though the grammar in
+  this document suggests otherwise. Also, not everything accepted by the 
grammar is necessarily valid CQL.
+- References to keywords or pieces of CQL code in running text will be shown 
in a ``fixed-width font``.
+
+
+.. _identifiers:
+
+Identifiers and keywords
+
+
+The CQL language uses *identifiers* (or *names*) to identify tables, columns 
and other objects. An identifier is a token
+matching the regular expression ``[a-zA-Z][a-zA-Z0-9_]*``.
+
+A number of such identifiers, like ``SELECT`` or ``WITH``, are *keywords*. 
They have a fixed meaning for the language
+and most are reserved. The list of those keywords can be found in 
:ref:`appendix-A`.
+
+Identifiers and (unquoted) keywords are case insensitive. Thus ``SELECT`` is 
the same than ``select`` or ``sElEcT``, and
+``myId`` is the same than ``myid`` or ``MYID``. A convention often used (in 
particular by the samples of this
+documentation) is to use upper case for keywords and lower case for other 
identifiers.
+
+There is a second kind of identifiers called *quoted identifiers* defined by 
enclosing an arbitrary sequence of
+characters (non empty) in double-quotes(``"``). Quoted identifiers are never 
keywords. Thus ``"select"`` is not a
+reserved keyword and can be used to refer to a column (note that using this is 
particularly advised), while ``select``
+would raise a parsing error. Also, contrarily to unquoted identifiers and 
keywords, quoted identifiers are case
+sensitive (``"My Quoted Id"`` is *different* from ``"my quoted id"``). A fully 
lowercase quoted identifier that matches
+``[a-zA-Z][a-zA-Z0-9_]*`` is however *equivalent* to the unquoted identifier 
obtained by removing the double-quote (so
+``"myid"`` is equivalent to ``myid`` and to ``myId`` but different from 
``"myId"``).  Inside a quoted identifier, the
+double-quote character can be repeated to escape it, so ``"foo "" bar"`` is a 
valid identifier.
+
+.. note:: *quoted identifiers* allows to declare columns with arbitrary names, 
and those can sometime clash with
+   specific names used by the server. For instance, when using conditional 
update, the server will respond with a
+   result-set containing a special result named ``"[applied]"``. If you’ve 
declared a column with such a name, this
+   could potentially confuse some tools and should be avoided. In general, 
unquoted identifiers should be preferred but
+   if you use quoted identifiers, it is strongly advised to avoid any name 
enclosed by squared brackets (like
+   ``"[applied]"``) and any name that looks like a function call (like 
``"f(x)"``).
+
+More formally, we have:
+
+.. productionlist::
+   identifier: `unquoted_identifier` | `quoted_identifier`
+   unquot

[19/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/getting_started.rst
--
diff --git a/doc/source/getting_started.rst b/doc/source/getting_started.rst
deleted file mode 100644
index b20376a..000
--- a/doc/source/getting_started.rst
+++ /dev/null
@@ -1,269 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
-.. or more contributor license agreements.  See the NOTICE file
-.. distributed with this work for additional information
-.. regarding copyright ownership.  The ASF licenses this file
-.. to you under the Apache License, Version 2.0 (the
-.. "License"); you may not use this file except in compliance
-.. with the License.  You may obtain a copy of the License at
-..
-.. http://www.apache.org/licenses/LICENSE-2.0
-..
-.. Unless required by applicable law or agreed to in writing, software
-.. distributed under the License is distributed on an "AS IS" BASIS,
-.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-.. See the License for the specific language governing permissions and
-.. limitations under the License.
-
-.. highlight:: none
-
-Getting Started
-===
-
-Installing Cassandra
-
-
-Prerequisites
-^
-
-- The latest version of Java 8, either the `Oracle Java Standard Edition 8
-  `__ or 
`OpenJDK 8 `__. To
-  verify that you have the correct version of java installed, type ``java 
-version``.
-
-- For using cqlsh, the latest version of `Python 2.7 
`__. To verify that you have
-  the correct version of Python installed, type ``python --version``.
-
-Installation from binary tarball files
-^^
-
-- Download the latest stable release from the `Apache Cassandra downloads 
website `__.
-
-- Untar the file somewhere, for example:
-
-::
-
-tar -xvf apache-cassandra-3.6-bin.tar.gz cassandra
-
-The files will be extracted into ``apache-cassandra-3.6``, you need to 
substitute 3.6 with the release number that you
-have downloaded.
-
-- Optionally add ``apache-cassandra-3.6\bin`` to your path.
-- Start Cassandra in the foreground by invoking ``bin/cassandra -f`` from the 
command line. Press "Control-C" to stop
-  Cassandra. Start Cassandra in the background by invoking ``bin/cassandra`` 
from the command line. Invoke ``kill pid``
-  or ``pkill -f CassandraDaemon`` to stop Cassandra, where pid is the 
Cassandra process id, which you can find for
-  example by invoking ``pgrep -f CassandraDaemon``.
-- Verify that Cassandra is running by invoking ``bin/nodetool status`` from 
the command line.
-- Configuration files are located in the ``conf`` sub-directory.
-- Since Cassandra 2.1, log and data directories are located in the ``logs`` 
and ``data`` sub-directories respectively.
-  Older versions defaulted to ``/var/log/cassandra`` and 
``/var/lib/cassandra``. Due to this, it is necessary to either
-  start Cassandra with root privileges or change ``conf/cassandra.yaml`` to 
use directories owned by the current user,
-  as explained below in the section on changing the location of directories.
-
-Installation from Debian packages
-^
-
-- Add the Apache repository of Cassandra to 
``/etc/apt/sources.list.d/cassandra.sources.list``, for example for version
-  3.6:
-
-::
-
-echo "deb http://www.apache.org/dist/cassandra/debian 36x main" | sudo tee 
-a /etc/apt/sources.list.d/cassandra.sources.list
-
-- Update the repositories:
-
-::
-
-sudo apt-get update
-
-- If you encounter this error:
-
-::
-
-GPG error: http://www.apache.org 36x InRelease: The following signatures 
couldn't be verified because the public key is not available: NO_PUBKEY 
749D6EEC0353B12C
-
-Then add the public key 749D6EEC0353B12C as follows:
-
-::
-
-gpg --keyserver pgp.mit.edu --recv-keys 749D6EEC0353B12C
-gpg --export --armor 749D6EEC0353B12C | sudo apt-key add -
-
-and repeat ``sudo apt-get update``. The actual key may be different, you get 
it from the error message itself. For a
-full list of Apache contributors public keys, you can refer to `this link 
`__.
-
-- Install Cassandra:
-
-::
-
-sudo apt-get install cassandra
-
-- You can start Cassandra with ``sudo service cassandra start`` and stop it 
with ``sudo service cassandra stop``.
-  However, normally the service will start automatically. For this reason be 
sure to stop it if you need to make any
-  configuration changes.
-- Verify that Cassandra is running by invoking ``nodetool status`` from the 
command line.
-- The default location of configuration files is ``/etc/cassandra``.
-- The default location of log and data directories is ``/var/log/cassandra/`` 
and ``/var/lib/cassandra``.
-
-Configurin

[28/34] cassandra git commit: Finish fixing the CQL doc

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e624238/doc/source/cql/functions.rst
--
diff --git a/doc/source/cql/functions.rst b/doc/source/cql/functions.rst
index cf52ace..efcdf32 100644
--- a/doc/source/cql/functions.rst
+++ b/doc/source/cql/functions.rst
@@ -16,109 +16,113 @@
 
 .. highlight:: sql
 
-.. _cql_functions:
+.. _cql-functions:
+
+.. Need some intro for UDF and native functions in general and point those to 
it.
+.. _udfs:
+.. _native-functions:
 
 Functions
 -
 
-CQL3 distinguishes between built-in functions (so called ‘native
-functions’) and `user-defined functions <#udfs>`__. CQL3 includes
-several native functions, described below:
+CQL supports 2 main categories of functions:
+
+- the :ref:`scalar functions `, which simply take a number 
of values and produce an output with it.
+- the :ref:`aggregate functions `, which are used to 
aggregate multiple rows results from a
+  ``SELECT`` statement.
+
+In both cases, CQL provides a number of native "hard-coded" functions as well 
as the ability to create new user-defined
+functions.
+
+.. note:: By default, the use of user-defined functions is disabled by default 
for security concerns (even when
+   enabled, the execution of user-defined functions is sandboxed and a "rogue" 
function should not be allowed to do
+   evil, but no sandbox is perfect so using user-defined functions is opt-in). 
See the ``enable_user_defined_functions``
+   in ``cassandra.yaml`` to enable them.
+
+.. _scalar-functions:
 
 Scalar functions
 
 
+.. _scalar-native-functions:
+
 Native functions
 
 
 Cast
 
 
-The ``cast`` function can be used to converts one native datatype to
-another.
-
-The following table describes the conversions supported by the ``cast``
-function. Cassandra will silently ignore any cast converting a datatype
-into its own datatype.
-
-+-+-+
-| from| to 
 |
-+=+=+
-| ``ascii``   | ``text``, ``varchar``  
 |
-+-+-+
-| ``bigint``  | ``tinyint``, ``smallint``, ``int``, ``float``, ``double``, 
``decimal``, ``varint``, ``text``, ``varchar``   |
-+-+-+
-| ``boolean`` | ``text``, ``varchar``  
 |
-+-+-+
-| ``counter`` | ``tinyint``, ``smallint``, ``int``, ``bigint``, ``float``, 
``double``, ``decimal``, ``varint``, ``text``, ``varchar``   |
-+-+-+
-| ``date``| ``timestamp``  
 |
-+-+-+
-| ``decimal`` | ``tinyint``, ``smallint``, ``int``, ``bigint``, ``float``, 
``double``, ``varint``, ``text``, ``varchar``|
-+-+-+
-| ``double``  | ``tinyint``, ``smallint``, ``int``, ``bigint``, ``float``, 
``decimal``, ``varint``, ``text``, ``varchar``   |
-+-+-+
-| ``float``   | ``tinyint``, ``smallint``, ``int``, ``bigint``, 
``double``, ``decimal``, ``varint``, ``text``, ``varchar``  |
-+-+-+
-| ``inet``| ``text``, ``varchar``  
 |
-+-+-+
-| ``int`` | ``tinyint

[33/34] cassandra git commit: Modify build file to include html doc in artifacts

[slebresne]

Modify build file to include html doc in artifacts


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5cefe503
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5cefe503
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5cefe503

Branch: refs/heads/trunk
Commit: 5cefe503e0b529ef3d6b81c467de93d39e77a97d
Parents: 7988b3f
Author: Sylvain Lebresne 
Authored: Mon Jun 27 20:31:43 2016 +0200
Committer: Sylvain Lebresne 
Committed: Mon Jun 27 20:31:43 2016 +0200

--
 build.xml | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5cefe503/build.xml
--
diff --git a/build.xml b/build.xml
index 51f7252..8946ea3 100644
--- a/build.xml
+++ b/build.xml
@@ -1021,7 +1021,7 @@
 
 
 
-
   
   
@@ -1041,9 +1041,15 @@
   
   
 
-  
+  
+  
+  
 
   
+  
+
+
+  
   
 
   

[31/34] cassandra git commit: Fix minor errors in CQL doc

[slebresne]

Fix minor errors in CQL doc


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/02d26214
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/02d26214
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/02d26214

Branch: refs/heads/trunk
Commit: 02d262146930c7dda7d2bebe9363a06de504c5aa
Parents: 0e62423
Author: Sylvain Lebresne 
Authored: Mon Jun 27 12:18:35 2016 +0200
Committer: Sylvain Lebresne 
Committed: Mon Jun 27 15:02:05 2016 +0200

--
 doc/source/cql/ddl.rst | 39 +
 doc/source/cql/definitions.rst |  6 +++---
 doc/source/cql/dml.rst | 13 +++--
 doc/source/cql/indexes.rst |  8 
 doc/source/cql/mvs.rst |  6 ++
 doc/source/cql/triggers.rst|  6 ++
 6 files changed, 52 insertions(+), 26 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/02d26214/doc/source/cql/ddl.rst
--
diff --git a/doc/source/cql/ddl.rst b/doc/source/cql/ddl.rst
index 948298b..7f3431a 100644
--- a/doc/source/cql/ddl.rst
+++ b/doc/source/cql/ddl.rst
@@ -50,6 +50,11 @@ Further, a table is always part of a keyspace and a table 
name can be provided f
 part of. If is is not fully-qualified, the table is assumed to be in the 
*current* keyspace (see :ref:`USE statement
 `).
 
+Further, the valid names for columns is simply defined as:
+
+.. productionlist::
+   column_name: `identifier`
+
 We also define the notion of statement options for use in the following 
section:
 
 .. productionlist::
@@ -164,15 +169,15 @@ Creating a new table uses the ``CREATE TABLE`` statement:
  : ( ',' `column_definition` )*
  : [ ',' PRIMARY KEY '(' `primary_key` ')' ]
  : ')' [ WITH `table_options` ]
-   column_definition: `identifier` `cql_type` [ STATIC ] [ PRIMARY KEY]
+   column_definition: `column_name` `cql_type` [ STATIC ] [ PRIMARY KEY]
primary_key: `partition_key` [ ',' `clustering_columns` ]
-   partition_key: `identifier`
-: | '(' `identifier` ( ',' `identifier` )* ')'
-   clustering_columns: `identifier` ( ',' `identifier` )*
+   partition_key: `column_name`
+: | '(' `column_name` ( ',' `column_name` )* ')'
+   clustering_columns: `column_name` ( ',' `column_name` )*
table_options: COMPACT STORAGE [ AND `table_options` ]
: | CLUSTERING ORDER BY '(' `clustering_order` ')' [ AND 
`table_options` ]
: | `options`
-   clustering_order: `identifier` (ASC | DESC) ( ',' `identifier` (ASC | DESC) 
)*
+   clustering_order: `column_name` (ASC | DESC) ( ',' `column_name` (ASC | 
DESC) )*
 
 For instance::
 
@@ -554,9 +559,9 @@ Altering an existing table uses the ``ALTER TABLE`` 
statement:
 
 .. productionlist::
alter_table_statement: ALTER TABLE `table_name` `alter_table_instruction`
-   alter_table_instruction: ALTER `identifier` TYPE `cql_type`
-  : | ADD `identifier` `cql_type` ( ',' `identifier` 
`cql_type` )*
-  : | DROP `identifier` ( `identifier` )*
+   alter_table_instruction: ALTER `column_name` TYPE `cql_type`
+  : | ADD `column_name` `cql_type` ( ',' `column_name` 
`cql_type` )*
+  : | DROP `column_name` ( `column_name` )*
   : | WITH `options`
 
 For instance::
@@ -631,15 +636,15 @@ CQL data types may be converted only as the following 
table.
 
 Clustering columns have stricter requirements, only the following conversions 
are allowed:
 
-++---+
-| Existing type  | Can be altered to |
-++===+
-| ascii, text, varchar   | blob  |
-++---+
-| ascii, varchar | text  |
-++---+
-| ascii, text| varchar   |
-++---+
+++--+
+| Existing type  | Can be altered to|
+++==+
+| ascii, text, varchar   | blob |
+++--+
+| ascii, varchar | text |
+++--+
+| ascii, text| varchar  |
+++--+
 
 .. _drop-table-statement:
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/02d26214/doc/source/cql/definitions.rst
--
diff --git a/doc/source/cql/definitions.rst b/doc/source/cql/def

[29/34] cassandra git commit: Finish fixing the CQL doc

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e624238/doc/source/cql/dml.rst
--
diff --git a/doc/source/cql/dml.rst b/doc/source/cql/dml.rst
index 4cf34eb..58fdb05 100644
--- a/doc/source/cql/dml.rst
+++ b/doc/source/cql/dml.rst
@@ -21,586 +21,478 @@
 Data Manipulation
 -
 
+This section describes the statements supported by CQL to insert, update, 
delete and query data.
+
+.. _select-statement:
+
 SELECT
 ^^
 
-*Syntax:*
+Querying data from data is done using a ``SELECT`` statement:
 
-| bc(syntax)..
-|  ::= SELECT ( JSON )? 
-|  FROM 
-|  ( WHERE )?
-|  ( ORDER BY )?
-|  ( PER PARTITION LIMIT )?
-|  ( LIMIT )?
-|  ( ALLOW FILTERING )?
+.. productionlist::
+   select_statement: SELECT [ JSON | DISTINCT ] ( `select_clause` | '*' )
+   : FROM `table_name`
+   : [ WHERE `where_clause` ]
+   : [ ORDER BY `ordering_clause` ]
+   : [ PER PARTITION LIMIT (`integer` | `bind_marker`) ]
+   : [ LIMIT (`integer` | `bind_marker`) ]
+   : [ ALLOW FILTERING ]
+   select_clause: `selector` [ AS `identifier` ] ( ',' `selector` [ AS 
`identifier` ] )
+   selector: `identifier`
+   : | `term`
+   : | CAST '(' `selector` AS `cql_type` ')'
+   : | `function_name` '(' [ `selector` ( ',' `selector` )* ] ')'
+   : | COUNT '(' '*' ')'
+   where_clause: `relation` ( AND `relation` )*
+   relation: `identifier` `operator` `term`
+   : '(' `identifier` ( ',' `identifier` )* ')' `operator` 
`tuple_literal`
+   : TOKEN '(' `identifier` ( ',' `identifier` )* ')' `operator` `term`
+   operator: '=' | '<' | '>' | '<=' | '>=' | '!=' | IN | CONTAINS | CONTAINS 
KEY
+   ordering_clause: `identifier` [ ASC | DESC ] ( ',' `identifier` [ ASC | 
DESC ] )*
 
-|  ::= DISTINCT? 
-|  \| COUNT ‘(’ ( ‘\*’ \| ‘1’ ) ‘)’ (AS )?
+For instance::
 
-|  ::= (AS )? ( ‘,’ (AS )? )\*
-|  \| ‘\*’
+SELECT name, occupation FROM users WHERE userid IN (199, 200, 207);
+SELECT JSON name, occupation FROM users WHERE userid = 199;
+SELECT name AS user_name, occupation AS user_occupation FROM users;
 
-|  ::= 
-|  \| WRITETIME ‘(’ ‘)’
-|  \| TTL ‘(’ ‘)’
-|  \| CAST ‘(’ AS ‘)’
-|  \| ‘(’ ( (‘,’ )\*)? ‘)’
+SELECT time, value
+FROM events
+WHERE event_type = 'myEvent'
+  AND time > '2011-02-03'
+  AND time <= '2012-01-01'
 
- ::= ( AND )\*
+SELECT COUNT (*) AS user_count FROM users;
 
-|  ::= 
-|  \| ‘(’ (‘,’ )\* ‘)’ 
-|  \| IN ‘(’ ( ( ‘,’ )\* )? ‘)’
-|  \| ‘(’ (‘,’ )\* ‘)’ IN ‘(’ ( ( ‘,’ )\* )? ‘)’
-|  \| TOKEN ‘(’ ( ‘,’ )\* ‘)’ 
+The ``SELECT`` statements reads one or more columns for one or more rows in a 
table. It returns a result-set of the rows
+matching the request, where each row contains the values for the selection 
corresponding to the query. Additionally,
+:ref:`functions ` including :ref:`aggregation 
` ones can be applied to the result.
 
-|  ::= ‘=’ \| ‘<’ \| ‘>’ \| ‘<=’ \| ‘>=’ \| CONTAINS \| 
CONTAINS KEY
-|  ::= ( ‘,’ )\*
-|  ::= ( ASC \| DESC )?
-|  ::= ‘(’ (‘,’ )\* ‘)’
-| p.
-| *Sample:*
+A ``SELECT`` statement contains at least a :ref:`selection clause 
` and the name of the table on which
+the selection is on (note that CQL does **not** joins or sub-queries and thus 
a select statement only apply to a single
+table). In most case, a select will also have a :ref:`where clause 
` and it can optionally have additional
+clauses to :ref:`order ` or :ref:`limit ` the 
results. Lastly, :ref:`queries that require
+filtering ` can be allowed if the ``ALLOW FILTERING`` flag is 
provided.
+
+.. _selection-clause:
 
-| bc(sample)..
-| SELECT name, occupation FROM users WHERE userid IN (199, 200, 207);
+Selection clause
+
 
-SELECT JSON name, occupation FROM users WHERE userid = 199;
+The :token:`select_clause` determines which columns needs to be queried and 
returned in the result-set, as well as any
+transformation to apply to this result before returning. It consists of a 
comma-separated list of *selectors* or,
+alternatively, of the wildcard character (``*``) to select all the columns 
defined in the table.
 
-SELECT name AS user\_name, occupation AS user\_occupation FROM users;
+Selectors
+`
 
-| SELECT time, value
-| FROM events
-| WHERE event\_type = ‘myEvent’
-|  AND time > ‘2011-02-03’
-|  AND time <= ‘2012-01-01’
+A :token:`selector` can be one of:
 
-SELECT COUNT (\*) FROM users;
+- A column name of the table selected, to retrieve the values for that column.
+- A term, which is usually used nested inside other selectors like functions 
(if a term is selected directly, then the
+  corresponding column of the result-set will simply have the value of this 
term for every row returned).
+- A 

[34/34] cassandra git commit: Merge branch 'doc_in_tree' into trunk

[slebresne]

Merge branch 'doc_in_tree' into trunk

* doc_in_tree:
  Modify build file to include html doc in artifacts
  Add 'report bug' section
  Fix minor errors in CQL doc
  Finish fixing the CQL doc
  Don't track auto-generated file
  Reorganize document
  Automatically generate docs for cassandra.yaml
  Fix CQL doc (incomplete)
  Add Metrics/Monitoring docs
  Add Change Data Capture documentation
  Docs for Memtable and SSTable architecture
  Add doc on compaction
  Add initial version of security section
  Fill in Replication, Tuneable Consistency sections
  Add docs for cqlsh
  Add snitch and range movements section on Operations
  Add initial in-tree documentation (very incomplete so far)


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c7b9401b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c7b9401b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c7b9401b

Branch: refs/heads/trunk
Commit: c7b9401bf81d1bca12a802306576805053c0212b
Parents: 48e4d5d 5cefe50
Author: Sylvain Lebresne 
Authored: Mon Jun 27 20:33:21 2016 +0200
Committer: Sylvain Lebresne 
Committed: Mon Jun 27 20:33:21 2016 +0200

--
 .gitignore  |   3 +
 build.xml   |  23 +-
 conf/cassandra.yaml | 222 +++---
 doc/Makefile| 256 +++
 doc/README.md   |  31 +
 doc/convert_yaml_to_rst.py  | 144 
 doc/cql3/CQL.textile|   4 +-
 doc/make.bat| 281 
 doc/source/_static/extra.css|  43 ++
 doc/source/_templates/indexcontent.html |  33 +
 doc/source/architecture/dynamo.rst  | 137 
 doc/source/architecture/guarantees.rst  |  20 +
 doc/source/architecture/index.rst   |  29 +
 doc/source/architecture/overview.rst|  20 +
 doc/source/architecture/storage_engine.rst  |  82 +++
 doc/source/bugs.rst |  31 +
 doc/source/conf.py  | 432 
 doc/source/configuration/index.rst  |  25 +
 doc/source/contactus.rst|  53 ++
 doc/source/cql/appendices.rst   | 308 +
 doc/source/cql/changes.rst  | 189 ++
 doc/source/cql/ddl.rst  | 677 +++
 doc/source/cql/definitions.rst  | 230 +++
 doc/source/cql/dml.rst  | 499 ++
 doc/source/cql/functions.rst| 553 +++
 doc/source/cql/index.rst|  47 ++
 doc/source/cql/indexes.rst  |  83 +++
 doc/source/cql/json.rst | 112 +++
 doc/source/cql/mvs.rst  | 166 +
 doc/source/cql/security.rst | 497 ++
 doc/source/cql/triggers.rst |  63 ++
 doc/source/cql/types.rst| 518 ++
 doc/source/data_modeling/index.rst  |  20 +
 doc/source/faq/index.rst|  20 +
 doc/source/getting_started/configuring.rst  |  67 ++
 doc/source/getting_started/drivers.rst  | 107 +++
 doc/source/getting_started/index.rst|  33 +
 doc/source/getting_started/installing.rst   |  99 +++
 doc/source/getting_started/querying.rst |  52 ++
 doc/source/index.rst|  40 ++
 doc/source/operating/backups.rst|  22 +
 doc/source/operating/bloom_filters.rst  |  65 ++
 doc/source/operating/cdc.rst|  89 +++
 doc/source/operating/compaction.rst | 432 
 doc/source/operating/compression.rst|  94 +++
 doc/source/operating/hardware.rst   |  87 +++
 doc/source/operating/hints.rst  |  22 +
 doc/source/operating/index.rst  |  38 ++
 doc/source/operating/metrics.rst| 619 +
 doc/source/operating/read_repair.rst|  22 +
 doc/source/operating/repair.rst |  22 +
 doc/source/operating/security.rst   | 410 +++
 doc/source/operating/snitch.rst |  78 +++
 doc/source/operating/topo_changes.rst   | 122 
 doc/source/tools/cqlsh.rst  | 455 +
 doc/source/tools/index.rst  |  26 +
 doc/source/tools/nodetool.rst   |  22 +
 doc/source/troubleshooting/index.rst|  20 +
 .../cql3/statements/CreateViewStatement.java|   7 +-
 59 files changed, 8816 insertions(+), 85 deletions(-)
--


http://git-wip-u

[30/34] cassandra git commit: Finish fixing the CQL doc

[slebresne]

Finish fixing the CQL doc


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0e624238
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0e624238
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0e624238

Branch: refs/heads/trunk
Commit: 0e624238c15558155a57bcfa7ada9b3b84067658
Parents: 7dfb25e
Author: Sylvain Lebresne 
Authored: Fri Jun 24 17:01:37 2016 +0200
Committer: Sylvain Lebresne 
Committed: Fri Jun 24 17:01:37 2016 +0200

--
 .gitignore  |   2 +-
 doc/convert_yaml_to_rst.py  |   2 +-
 doc/source/cql/appendices.rst   |   2 -
 doc/source/cql/changes.rst  | 242 ++---
 doc/source/cql/ddl.rst  | 232 +++--
 doc/source/cql/definitions.rst  |  27 +-
 doc/source/cql/dml.rst  | 964 ---
 doc/source/cql/functions.rst| 890 -
 doc/source/cql/index.rst|   2 -
 doc/source/cql/indexes.rst  |  93 +-
 doc/source/cql/json.rst | 168 ++--
 doc/source/cql/mvs.rst  | 183 ++--
 doc/source/cql/security.rst | 768 ++-
 doc/source/cql/triggers.rst |  48 +-
 doc/source/cql/types.rst|   6 +-
 doc/source/operating/compaction.rst |   8 +-
 doc/source/operating/security.rst   |  10 +-
 .../cql3/statements/CreateViewStatement.java|   9 +-
 18 files changed, 1626 insertions(+), 2030 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e624238/.gitignore
--
diff --git a/.gitignore b/.gitignore
index 4f33eda..f5b1ce1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -74,4 +74,4 @@ lib/jsr223/scala/*.jar
 /.ant-targets-build.xml
 
 # Generated files from the documentation
-doc/source/cassandra_config_file.rst
+doc/source/configuration/cassandra_config_file.rst

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e624238/doc/convert_yaml_to_rst.py
--
diff --git a/doc/convert_yaml_to_rst.py b/doc/convert_yaml_to_rst.py
index 398295d..fee6d8c 100644
--- a/doc/convert_yaml_to_rst.py
+++ b/doc/convert_yaml_to_rst.py
@@ -59,7 +59,7 @@ def convert(yaml_file, dest_file):
 
 with open(dest_file, 'w') as outfile:
 outfile.write("Cassandra Configuration File\n")
-outfile.write("=\n")
+outfile.write("\n")
 
 # since comments preceed an option, this holds all of the comment
 # lines we've seen since the last option

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e624238/doc/source/cql/appendices.rst
--
diff --git a/doc/source/cql/appendices.rst b/doc/source/cql/appendices.rst
index 9788a56..c4bb839 100644
--- a/doc/source/cql/appendices.rst
+++ b/doc/source/cql/appendices.rst
@@ -300,8 +300,6 @@ names as their name.
 +-+
 | ``complex`` |
 +-+
-| ``date``|
-+-+
 | ``enum``|
 +-+
 | ``interval``|

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e624238/doc/source/cql/changes.rst
--
diff --git a/doc/source/cql/changes.rst b/doc/source/cql/changes.rst
index 9549f8f..263df13 100644
--- a/doc/source/cql/changes.rst
+++ b/doc/source/cql/changes.rst
@@ -24,234 +24,166 @@ The following describes the changes in each version of 
CQL.
 3.4.2
 ^
 
--  ```INSERT/UPDATE options`` <#updateOptions>`__ for tables having a
-   default\_time\_to\_live specifying a TTL of 0 will remove the TTL
-   from the inserted or updated values
--  ```ALTER TABLE`` <#alterTableStmt>`__ ``ADD`` and ``DROP`` now allow
-   mutiple columns to be added/removed
--  New ```PER PARTITION LIMIT`` <#selectLimit>`__ option (see
-   `CASSANDRA-7017 `__.
--  `User-defined functions <#udfs>`__ can now instantiate ``UDTValue``
-   and ``TupleValue`` instances via the new ``UDFContext`` interface
-   (see
-   `CASSANDRA-10818 
`__.
--  “User-defined types”#createTypeStmt may now be stored in a non-frozen
-   form, allowing individual fields to be updated and deleted in
-   ```UPDATE`` statements <#updateStmt>`__ and ```DELETE``
-   statements <#deleteStmt>`__, respectively.
-   (`CASSANDRA-7423 `__
+- If a table has a non zer

[10/34] cassandra git commit: Docs for Memtable and SSTable architecture

[slebresne]

Docs for Memtable and SSTable architecture


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7bf837ca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7bf837ca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7bf837ca

Branch: refs/heads/trunk
Commit: 7bf837cae0a24f72428becb739102521042ed0ad
Parents: 8d2bd0d
Author: Tyler Hobbs 
Authored: Fri Jun 17 15:10:09 2016 -0500
Committer: Sylvain Lebresne 
Committed: Tue Jun 21 14:12:59 2016 +0200

--
 doc/source/architecture.rst | 53 ++--
 1 file changed, 51 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7bf837ca/doc/source/architecture.rst
--
diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst
index 3f8a8ca..cb52477 100644
--- a/doc/source/architecture.rst
+++ b/doc/source/architecture.rst
@@ -145,20 +145,69 @@ throughput, latency, and availability.
 Storage Engine
 --
 
+.. _commit-log:
+
 CommitLog
 ^
 
 .. todo:: todo
 
+.. _memtables:
+
 Memtables
 ^
 
-.. todo:: todo
+Memtables are in-memory structures where Cassandra buffers writes.  In 
general, there is one active memtable per table.
+Eventually, memtables are flushed onto disk and become immutable `SSTables`_.  
This can be triggered in several
+ways:
+
+- The memory usage of the memtables exceeds the configured threshold  (see 
``memtable_cleanup_threshold``)
+- The :ref:`commit-log` approaches its maximum size, and forces memtable 
flushes in order to allow commitlog segments to
+  be freed
+
+Memtables may be stored entirely on-heap or partially off-heap, depending on 
``memtable_allocation_type``.
 
 SSTables
 
 
-.. todo:: todo
+SSTables are the immutable data files that Cassandra uses for persisting data 
on disk.
+
+As SSTables are flushed to disk from :ref:`memtables` or are streamed from 
other nodes, Cassandra triggers compactions
+which combine multiple SSTables into one.  Once the new SSTable has been 
written, the old SSTables can be removed.
+
+Each SSTable is comprised of multiple components stored in separate files:
+
+``Data.db``
+  The actual data, i.e. the contents of rows.
+
+``Index.db``
+  An index from partition keys to positions in the ``Data.db`` file.  For wide 
partitions, this may also include an
+  index to rows within a partition.
+
+``Summary.db``
+  A sampling of (by default) every 128th entry in the ``Index.db`` file.
+
+``Filter.db``
+  A Bloom Filter of the partition keys in the SSTable.
+
+``CompressionInfo.db``
+  Metadata about the offsets and lengths of compression chunks in the 
``Data.db`` file.
+
+``Statistics.db``
+  Stores metadata about the SSTable, including information about timestamps, 
tombstones, clustering keys, compaction,
+  repair, compression, TTLs, and more.
+
+``Digest.crc32``
+  A CRC-32 digest of the ``Data.db`` file.
+
+``TOC.txt``
+  A plain text list of the component files for the SSTable.
+
+Within the ``Data.db`` file, rows are organized by partition.  These 
partitions are sorted in token order (i.e. by a
+hash of the partition key when the default partitioner, ``Murmur3Partition``, 
is used).  Within a partition, rows are
+stored in the order of their clustering keys.
+
+SSTables can be optionally compressed using block-based compression.
 
 Guarantees
 --

[22/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/cql/appendices.rst
--
diff --git a/doc/source/cql/appendices.rst b/doc/source/cql/appendices.rst
new file mode 100644
index 000..9788a56
--- /dev/null
+++ b/doc/source/cql/appendices.rst
@@ -0,0 +1,310 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. highlight:: sql
+
+Appendices
+--
+
+.. _appendix-A:
+
+Appendix A: CQL Keywords
+
+
+CQL distinguishes between *reserved* and *non-reserved* keywords.
+Reserved keywords cannot be used as identifier, they are truly reserved
+for the language (but one can enclose a reserved keyword by
+double-quotes to use it as an identifier). Non-reserved keywords however
+only have a specific meaning in certain context but can used as
+identifier otherwise. The only *raison d’être* of these non-reserved
+keywords is convenience: some keyword are non-reserved when it was
+always easy for the parser to decide whether they were used as keywords
+or not.
+
+++-+
+| Keyword| Reserved?   |
+++=+
+| ``ADD``| yes |
+++-+
+| ``AGGREGATE``  | no  |
+++-+
+| ``ALL``| no  |
+++-+
+| ``ALLOW``  | yes |
+++-+
+| ``ALTER``  | yes |
+++-+
+| ``AND``| yes |
+++-+
+| ``APPLY``  | yes |
+++-+
+| ``AS`` | no  |
+++-+
+| ``ASC``| yes |
+++-+
+| ``ASCII``  | no  |
+++-+
+| ``AUTHORIZE``  | yes |
+++-+
+| ``BATCH``  | yes |
+++-+
+| ``BEGIN``  | yes |
+++-+
+| ``BIGINT`` | no  |
+++-+
+| ``BLOB``   | no  |
+++-+
+| ``BOOLEAN``| no  |
+++-+
+| ``BY`` | yes |
+++-+
+| ``CALLED`` | no  |
+++-+
+| ``CLUSTERING`` | no  |
+++-+
+| ``COLUMNFAMILY``   | yes |
+++-+
+| ``COMPACT``| no  |
+++-+
+| ``CONTAINS``   | no  |
+++-+
+| ``COUNT``  | no  |
+++-+
+| ``COUNTER``| no  |
+++-+
+| ``CREATE`` | yes |
+++-+
+| ``CUSTOM`` | no  |
+++-+
+| ``DATE``   | no  |
+++-+
+| ``DECIMAL``| no  |
+++-+
+| ``DELETE`` | yes |
+++-+
+| ``DESC``   | yes |
+++-+
+| ``DESCRIBE``   | yes |
+++-+
+| ``DISTINCT``   | no  |
+++-+
+| ``DOUBLE`` | no  |
+++-+
+| ``DROP``   | yes |
+++-+
+| ``ENTRIES``| yes |
+++-+
+| ``EXECUTE``| yes |
+++-+
+| ``EXISTS`` | no  |
+++-+
+| ``FILTERING``  | no  |
+++-+
+| ``FINALFUNC``  | no  |
+++-+
+| ``FLOAT``  | no  |
+++---

[32/34] cassandra git commit: Add 'report bug' section

[slebresne]

Add 'report bug' section


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7988b3fc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7988b3fc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7988b3fc

Branch: refs/heads/trunk
Commit: 7988b3fcda4d7948f510b540a1c51fff3c910722
Parents: 02d2621
Author: Sylvain Lebresne 
Authored: Mon Jun 27 20:01:31 2016 +0200
Committer: Sylvain Lebresne 
Committed: Mon Jun 27 20:01:31 2016 +0200

--
 doc/source/bugs.rst | 17 ++---
 doc/source/contactus.rst|  4 
 doc/source/cql/index.rst|  2 ++
 doc/source/getting_started/drivers.rst  |  2 ++
 doc/source/getting_started/querying.rst | 22 ++
 5 files changed, 40 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7988b3fc/doc/source/bugs.rst
--
diff --git a/doc/source/bugs.rst b/doc/source/bugs.rst
index ef10aab..90efb14 100644
--- a/doc/source/bugs.rst
+++ b/doc/source/bugs.rst
@@ -14,7 +14,18 @@
 .. See the License for the specific language governing permissions and
 .. limitations under the License.
 
-Reporting bugs
---
+Reporting bugs and contributing
+===
 
-.. todo:: TODO
+If you encounter a problem with Cassandra, the first places to ask for help 
are the :ref:`user mailing list
+` and the ``#cassandra`` :ref:`IRC channel `.
+
+If, after having asked for help, you suspect that you have found a bug in 
Cassandra, you should report it by opening a
+ticket through the `Apache Cassandra JIRA 
`__. Please provide as much
+details as you can on your problem, and don't forget to indicate which version 
of Cassandra you are running and on which
+environment.
+
+If you would like to contribute, please check `the section on contributing
+`__ on the Cassandra wiki. 
Please note that the source of this
+documentation is part of the Cassandra git repository and hence contributions 
to the documentation should follow the
+same path.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7988b3fc/doc/source/contactus.rst
--
diff --git a/doc/source/contactus.rst b/doc/source/contactus.rst
index ffd9d60..8d0f5dd 100644
--- a/doc/source/contactus.rst
+++ b/doc/source/contactus.rst
@@ -19,6 +19,8 @@ Contact us
 
 You can get in touch with the Cassandra community either via the mailing lists 
or the freenode IRC channels.
 
+.. _mailing-lists:
+
 Mailing lists
 -
 
@@ -37,6 +39,8 @@ Subscribe by sending an email to the email address in the 
Subscribe links above.
 email to confirm your subscription. Make sure to keep the welcome email as it 
contains instructions on how to
 unsubscribe.
 
+.. _irc-channels:
+
 IRC
 ---
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7988b3fc/doc/source/cql/index.rst
--
diff --git a/doc/source/cql/index.rst b/doc/source/cql/index.rst
index 718959c..00d90e4 100644
--- a/doc/source/cql/index.rst
+++ b/doc/source/cql/index.rst
@@ -14,6 +14,8 @@
 .. See the License for the specific language governing permissions and
 .. limitations under the License.
 
+.. _cql:
+
 The Cassandra Query Language (CQL)
 ==
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7988b3fc/doc/source/getting_started/drivers.rst
--
diff --git a/doc/source/getting_started/drivers.rst 
b/doc/source/getting_started/drivers.rst
index d637a70..baec823 100644
--- a/doc/source/getting_started/drivers.rst
+++ b/doc/source/getting_started/drivers.rst
@@ -14,6 +14,8 @@
 .. See the License for the specific language governing permissions and
 .. limitations under the License.
 
+.. _client-drivers:
+
 Client drivers
 --
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7988b3fc/doc/source/getting_started/querying.rst
--
diff --git a/doc/source/getting_started/querying.rst 
b/doc/source/getting_started/querying.rst
index f73c106..55b162b 100644
--- a/doc/source/getting_started/querying.rst
+++ b/doc/source/getting_started/querying.rst
@@ -17,9 +17,18 @@
 Inserting and querying
 --
 
-cqlsh is a command line shell for interacting with Cassandra through CQL (the 
Cassandra Query Language). It is shipped
-with every Cassandra package, and can be found in the bin/ directory alongside 
the cassandra executable. cqlsh utilizes
-the Pyth

[26/34] cassandra git commit: Don't track auto-generated file

[slebresne]

Don't track auto-generated file


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7dfb25e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7dfb25e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7dfb25e3

Branch: refs/heads/trunk
Commit: 7dfb25e3c29c46c5ba58c63613c8cc3b4ca28412
Parents: 54f7335
Author: Sylvain Lebresne 
Authored: Wed Jun 22 10:06:09 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jun 22 10:06:09 2016 +0200

--
 doc/Makefile|6 +-
 .../configuration/cassandra_config_file.rst | 1699 --
 2 files changed, 5 insertions(+), 1700 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dfb25e3/doc/Makefile
--
diff --git a/doc/Makefile b/doc/Makefile
index 14e4c7a..9a736cc 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -14,7 +14,10 @@ ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees 
$(PAPEROPT_$(PAPER)) $(SPHINXOPTS) sou
 # the i18n builder cannot share the environment and doctrees with the others
 I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
 
-MAKE_CASSANDRA_YAML = python convert_yaml_to_rst.py ../conf/cassandra.yaml 
source/configuration/cassandra_config_file.rst
+YAML_DOC_INPUT=../conf/cassandra.yaml
+YAML_DOC_OUTPUT=source/configuration/cassandra_config_file.rst
+
+MAKE_CASSANDRA_YAML = python convert_yaml_to_rst.py $(YAML_DOC_INPUT) 
$(YAML_DOC_OUTPUT)
 
 .PHONY: help
 help:
@@ -49,6 +52,7 @@ help:
 .PHONY: clean
 clean:
rm -rf $(BUILDDIR)/*
+   rm $(YAML_DOC_OUTPUT)
 
 .PHONY: html
 html:

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7dfb25e3/doc/source/configuration/cassandra_config_file.rst
--
diff --git a/doc/source/configuration/cassandra_config_file.rst 
b/doc/source/configuration/cassandra_config_file.rst
deleted file mode 100644
index b7d1bbc..000
--- a/doc/source/configuration/cassandra_config_file.rst
+++ /dev/null
@@ -1,1699 +0,0 @@
-Cassandra Configuration File
-=
-
-``cluster_name``
-
-The name of the cluster. This is mainly used to prevent machines in
-one logical cluster from joining another.
-
-*Default Value:* 'Test Cluster'
-
-``num_tokens``
---
-
-This defines the number of tokens randomly assigned to this node on the ring
-The more tokens, relative to other nodes, the larger the proportion of data
-that this node will store. You probably want all nodes to have the same number
-of tokens assuming they have equal hardware capability.
-
-If you leave this unspecified, Cassandra will use the default of 1 token for 
legacy compatibility,
-and will use the initial_token as described below.
-
-Specifying initial_token will override this setting on the node's initial 
start,
-on subsequent starts, this setting will apply even if initial token is set.
-
-If you already have a cluster with 1 token per node, and wish to migrate to 
-multiple tokens per node, see http://wiki.apache.org/cassandra/Operations
-
-*Default Value:* 256
-
-``allocate_tokens_for_keyspace``
-
-*This option is commented out by default.*
-
-Triggers automatic allocation of num_tokens tokens for this node. The 
allocation
-algorithm attempts to choose tokens in a way that optimizes replicated load 
over
-the nodes in the datacenter for the replication strategy used by the specified
-keyspace.
-
-The load assigned to each node will be close to proportional to its number of
-vnodes.
-
-Only supported with the Murmur3Partitioner.
-
-*Default Value:* KEYSPACE
-
-``initial_token``
--
-*This option is commented out by default.*
-
-initial_token allows you to specify tokens manually.  While you can use it with
-vnodes (num_tokens > 1, above) -- in which case you should provide a 
-comma-separated list -- it's primarily used when adding nodes to legacy 
clusters 
-that do not have vnodes enabled.
-
-``hinted_handoff_enabled``
---
-
-See http://wiki.apache.org/cassandra/HintedHandoff
-May either be "true" or "false" to enable globally
-
-*Default Value:* true
-
-``hinted_handoff_disabled_datacenters``

-*This option is commented out by default.*
-
-When hinted_handoff_enabled is true, a black list of data centers that will not
-perform hinted handoff
-
-*Default Value (complex option)*::
-
-#- DC1
-#- DC2
-
-``max_hint_window_in_ms``
--
-this defines the maximum amount of time a dead host will have hints
-generated.  After it has been dead this long, new hints for it will not be
-created until it has been seen alive and gone down again.
-
-*

[25/34] cassandra git commit: Reorganize document

[slebresne]

Reorganize document


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54f7335c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54f7335c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54f7335c

Branch: refs/heads/trunk
Commit: 54f7335cd9bcf13f96d62191b64ea179a61c1763
Parents: f2f3071
Author: Sylvain Lebresne 
Authored: Tue Jun 21 19:52:52 2016 +0200
Committer: Sylvain Lebresne 
Committed: Wed Jun 22 09:30:46 2016 +0200

--
 .gitignore  |3 +
 doc/Makefile|2 +-
 doc/convert_yaml_to_rst.py  |2 +-
 doc/source/_static/extra.css|8 +
 doc/source/_templates/indexcontent.html |   33 +
 doc/source/architecture.rst |  217 -
 doc/source/architecture/dynamo.rst  |  137 +
 doc/source/architecture/guarantees.rst  |   20 +
 doc/source/architecture/index.rst   |   29 +
 doc/source/architecture/overview.rst|   20 +
 doc/source/architecture/storage_engine.rst  |   82 +
 doc/source/bugs.rst |   20 +
 doc/source/conf.py  |4 +-
 .../configuration/cassandra_config_file.rst | 1699 
 doc/source/configuration/index.rst  |   25 +
 doc/source/cql.rst  | 4114 --
 doc/source/cql/appendices.rst   |  310 ++
 doc/source/cql/changes.rst  |  257 ++
 doc/source/cql/ddl.rst  |  682 +++
 doc/source/cql/definitions.rst  |  225 +
 doc/source/cql/dml.rst  |  606 +++
 doc/source/cql/functions.rst|  661 +++
 doc/source/cql/index.rst|   47 +
 doc/source/cql/indexes.rst  |   84 +
 doc/source/cql/json.rst |  146 +
 doc/source/cql/mvs.rst  |   95 +
 doc/source/cql/security.rst |  637 +++
 doc/source/cql/triggers.rst |   61 +
 doc/source/cql/types.rst|  516 +++
 doc/source/cqlsh.rst|  447 --
 doc/source/data_modeling/index.rst  |   20 +
 doc/source/faq.rst  |   20 -
 doc/source/faq/index.rst|   20 +
 doc/source/getting_started.rst  |  269 --
 doc/source/getting_started/configuring.rst  |   67 +
 doc/source/getting_started/drivers.rst  |  105 +
 doc/source/getting_started/index.rst|   33 +
 doc/source/getting_started/installing.rst   |   99 +
 doc/source/getting_started/querying.rst |   38 +
 doc/source/index.rst|   19 +-
 doc/source/operating/backups.rst|   22 +
 doc/source/operating/bloom_filters.rst  |   65 +
 doc/source/operating/cdc.rst|   89 +
 doc/source/operating/compaction.rst |  426 ++
 doc/source/operating/compression.rst|   94 +
 doc/source/operating/hardware.rst   |   87 +
 doc/source/operating/hints.rst  |   22 +
 doc/source/operating/index.rst  |   38 +
 doc/source/operating/metrics.rst|  619 +++
 doc/source/operating/read_repair.rst|   22 +
 doc/source/operating/repair.rst |   22 +
 doc/source/operating/security.rst   |  410 ++
 doc/source/operating/snitch.rst |   78 +
 doc/source/operating/topo_changes.rst   |  122 +
 doc/source/operations.rst   | 1900 
 doc/source/tools/cqlsh.rst  |  455 ++
 doc/source/tools/index.rst  |   26 +
 doc/source/tools/nodetool.rst   |   22 +
 doc/source/troubleshooting.rst  |   20 -
 doc/source/troubleshooting/index.rst|   20 +
 60 files changed, 9440 insertions(+), 6998 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/.gitignore
--
diff --git a/.gitignore b/.gitignore
index 9cb8614..4f33eda 100644
--- a/.gitignore
+++ b/.gitignore
@@ -72,3 +72,6 @@ lib/jsr223/jython/cachedir
 lib/jsr223/scala/*.jar
 
 /.ant-targets-build.xml
+
+# Generated files from the documentation
+doc/source/cassandra_config_file.rst

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/Makefile
--
diff --git a/doc/Makefile b/doc/Makefile
index 778448a..14e4c7a 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -14,7 +14,7 @@ ALLSPHINXOPTS   = -d $(BUILDDIR)/doctr

[02/34] cassandra git commit: Add initial in-tree documentation (very incomplete so far)

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cad277be/doc/source/cql.rst
--
diff --git a/doc/source/cql.rst b/doc/source/cql.rst
new file mode 100644
index 000..8d36258
--- /dev/null
+++ b/doc/source/cql.rst
@@ -0,0 +1,3916 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. highlight:: sql
+
+The Cassandra Query Language (CQL)
+==
+
+CQL Syntax
+--
+
+Preamble
+
+
+This document describes the Cassandra Query Language (CQL) [#]_. Note that 
this document describes the last version of
+the languages. However, the `changes <#changes>`_ section provides the diff 
between the different versions of CQL.
+
+CQL offers a model close to SQL in the sense that data is put in *tables* 
containing *rows* of *columns*. For
+that reason, when used in this document, these terms (tables, rows and 
columns) have the same definition than they have
+in SQL. But please note that as such, they do **not** refer to the concept of 
rows and columns found in the deprecated
+thrift API (and earlier version 1 and 2 of CQL).
+
+Conventions
+^^^
+
+To aid in specifying the CQL syntax, we will use the following conventions in 
this document:
+
+- Language rules will be given in an informal `BNF variant
+  `_ notation. 
In particular, we'll use square brakets
+  (``[ item ]``) for optional items, ``*`` and ``+`` for repeated items (where 
``+`` imply at least one).
+- The grammar is provided for documentation purposes and leave some minor 
details out (only conveniences that can be
+  ignored). For instance, the comma on the last column definition in a 
``CREATE TABLE`` statement is optional but
+  supported if present even though the grammar in this document suggests 
otherwise. Also, not everything accepted by the
+  grammar will be valid CQL.
+- References to keywords or pieces of CQL code in running text will be shown 
in a ``fixed-width font``.
+
+Identifiers and keywords
+
+
+The CQL language uses *identifiers* (or *names*) to identify tables, columns 
and other objects. An identifier is a token
+matching the regular expression ``[a-zA-Z][a-zA-Z0-9_]*``.
+
+A number of such identifiers, like ``SELECT`` or ``WITH``, are *keywords*. 
They have a fixed meaning for the language
+and most are reserved. The list of those keywords can be found in `Appendix A 
<#appendixA>`__.
+
+Identifiers and (unquoted) keywords are case insensitive. Thus ``SELECT`` is 
the same than ``select`` or ``sElEcT``, and
+``myId`` is the same than ``myid`` or ``MYID``. A convention often used (in 
particular by the samples of this
+documentation) is to use upper case for keywords and lower case for other 
identifiers.
+
+There is a second kind of identifiers called *quoted identifiers* defined by 
enclosing an arbitrary sequence of
+characters (non empty) in double-quotes(``"``). Quoted identifiers are never 
keywords. Thus ``"select"`` is not a
+reserved keyword and can be used to refer to a column (note that using this is 
particularly advised), while ``select``
+would raise a parsing error. Also, contrarily to unquoted identifiers and 
keywords, quoted identifiers are case
+sensitive (``"My Quoted Id"`` is *different* from ``"my quoted id"``). A fully 
lowercase quoted identifier that matches
+``[a-zA-Z][a-zA-Z0-9_]*`` is however *equivalent* to the unquoted identifier 
obtained by removing the double-quote (so
+``"myid"`` is equivalent to ``myid`` and to ``myId`` but different from 
``"myId"``).  Inside a quoted identifier, the
+double-quote character can be repeated to escape it, so ``"foo "" bar"`` is a 
valid identifier.
+
+**Warning**: *quoted identifiers* allows to declare columns with arbitrary 
names, and those can sometime clash with
+specific names used by the server. For instance, when using conditional 
update, the server will respond with a
+result-set containing a special result named ``"[applied]"``. If you’ve 
declared a column with such a name, this could
+potentially confuse some tools and should be avoided. In general, unquoted 
identifiers should be preferred

[12/34] cassandra git commit: Fix CQL doc (incomplete)

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d65542b/doc/source/operations.rst
--
diff --git a/doc/source/operations.rst b/doc/source/operations.rst
index 31ecc35..cf8812f 100644
--- a/doc/source/operations.rst
+++ b/doc/source/operations.rst
@@ -19,11 +19,6 @@
 Operating Cassandra
 ===
 
-Replication Strategies
---
-
-.. todo:: todo
-
 Snitch
 --
 
@@ -264,12 +259,12 @@ tombstone that has not been propagated to all replicas 
and that could cause dele
 
 To be able to drop an actual tombstone the following needs to be true;
 
--  The tombstone must be older than ``gc_grace_seconds``
--  If partition X contains the tombstone, the sstable containing the partition 
plus all sstables containing data older
+- The tombstone must be older than ``gc_grace_seconds``
+- If partition X contains the tombstone, the sstable containing the partition 
plus all sstables containing data older
   than the tombstone containing X must be included in the same compaction. We 
don't need to care if the partition is in
   an sstable if we can guarantee that all data in that sstable is newer than 
the tombstone. If the tombstone is older
   than the data it cannot shadow that data.
--  If the option ``only_purge_repaired_tombstones`` is enabled, tombstones are 
only removed if the data has also been
+- If the option ``only_purge_repaired_tombstones`` is enabled, tombstones are 
only removed if the data has also been
   repaired.
 
 TTL
@@ -538,9 +533,9 @@ The primary motivation for TWCS is to separate data on disk 
by timestamp and to
 more efficiently. One potential way this optimal behavior can be subverted is 
if data is written to SSTables out of
 order, with new data and old data in the same SSTable. Out of order data can 
appear in two ways:
 
--  If the user mixes old data and new data in the traditional write path, the 
data will be comingled in the memtables
+- If the user mixes old data and new data in the traditional write path, the 
data will be comingled in the memtables
   and flushed into the same SSTable, where it will remain comingled.
--  If the user's read requests for old data cause read repairs that pull old 
data into the current memtable, that data
+- If the user's read requests for old data cause read repairs that pull old 
data into the current memtable, that data
   will be comingled and flushed into the same SSTable.
 
 While TWCS tries to minimize the impact of comingled data, users should 
attempt to avoid this behavior.  Specifically,

[08/34] cassandra git commit: Add doc on compaction

[slebresne]

Add doc on compaction


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d2bd0d9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d2bd0d9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d2bd0d9

Branch: refs/heads/trunk
Commit: 8d2bd0d9cecd2ced794a65642a74f1f422696614
Parents: 898b91a
Author: Marcus Eriksson 
Authored: Fri Jun 17 22:03:25 2016 +0200
Committer: Sylvain Lebresne 
Committed: Fri Jun 17 22:03:38 2016 +0200

--
 doc/source/operations.rst | 360 +++--
 1 file changed, 348 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d2bd0d9/doc/source/operations.rst
--
diff --git a/doc/source/operations.rst b/doc/source/operations.rst
index 9e79700..9094766 100644
--- a/doc/source/operations.rst
+++ b/doc/source/operations.rst
@@ -205,26 +205,362 @@ Hints
 
 .. todo:: todo
 
+.. _compaction:
+
 Compaction
 --
 
-Size Tiered
-^^^
+Types of compaction
+^^^
 
-.. todo:: todo
+The concept of compaction is used for different kinds of operations in 
Cassandra, the common thing about these
+operations is that it takes one or more sstables and output new sstables. The 
types of compactions are;
+
+Minor compaction
+triggered automatically in Cassandra.
+Major compaction
+a user executes a compaction over all sstables on the node.
+User defined compaction
+a user triggers a compaction on a given set of sstables.
+Scrub
+try to fix any broken sstables. This can actually remove valid data if 
that data is corrupted, if that happens you
+will need to run a full repair on the node.
+Upgradesstables
+upgrade sstables to the latest version. Run this after upgrading to a new 
major version.
+Cleanup
+remove any ranges this node does not own anymore, typically triggered on 
neighbouring nodes after a node has been
+bootstrapped since that node will take ownership of some ranges from those 
nodes.
+Secondary index rebuild
+rebuild the secondary indexes on the node.
+Anticompaction
+after repair the ranges that were actually repaired are split out of the 
sstables that existed when repair started.
+
+When is a minor compaction triggered?
+^
+
+#  When an sstable is added to the node through flushing/streaming etc.
+#  When autocompaction is enabled after being disabled (``nodetool 
enableautocompaction``)
+#  When compaction adds new sstables.
+#  A check for new minor compactions every 5 minutes.
+
+Merging sstables
+
 
-Leveled
-^^^
+Compaction is about merging sstables, since partitions in sstables are sorted 
based on the hash of the partition key it
+is possible to efficiently merge separate sstables. Content of each partition 
is also sorted so each partition can be
+merged efficiently.
 
-.. todo:: todo
+Tombstones and gc_grace
+^^^
 
-TimeWindow
-^^
-.. todo:: todo
+When a delete is issued in Cassandra what happens is that a tombstone is 
written, this tombstone shadows the data it
+deletes. This means that the tombstone can live in one sstable and the data it 
covers is in another sstable. To be able
+to remove the actual data, a compaction where both the sstable containing the 
tombstone and the sstable containing the
+data is included the same compaction is needed.
+
+``gc_grace_seconds`` is the minimum time tombstones are kept around. If you 
generally run repair once a week, then
+``gc_grace_seconds`` needs to be at least 1 week (you probably want some 
margin as well), otherwise you might drop a
+tombstone that has not been propagated to all replicas and that could cause 
deleted data to become live again.
+
+To be able to drop an actual tombstone the following needs to be true;
+
+-  The tombstone must be older than ``gc_grace_seconds``
+-  If partition X contains the tombstone, the sstable containing the partition 
plus all sstables containing data older
+  than the tombstone containing X must be included in the same compaction. We 
don't need to care if the partition is in
+  an sstable if we can guarantee that all data in that sstable is newer than 
the tombstone. If the tombstone is older
+  than the data it cannot shadow that data.
+-  If the option ``only_purge_repaired_tombstones`` is enabled, tombstones are 
only removed if the data has also been
+  repaired.
+
+TTL
+^^^
+
+Data in Cassandra can have an additional property called time to live - this 
is used to automatically drop data that has
+expired once the time is reached. Once the TTL has expired the data is 
converted to a tombstone which stays around for
+at least ``gc_grace_seconds``. Note that if you mix data with

[01/34] cassandra git commit: Add initial in-tree documentation (very incomplete so far)

[slebresne]

Repository: cassandra
Updated Branches:
  refs/heads/trunk 48e4d5dae -> c7b9401bf


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cad277be/doc/source/faq.rst
--
diff --git a/doc/source/faq.rst b/doc/source/faq.rst
new file mode 100644
index 000..4ac0be4
--- /dev/null
+++ b/doc/source/faq.rst
@@ -0,0 +1,20 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+Frequently Asked Questions
+==
+
+.. TODO: todo

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cad277be/doc/source/getting_started.rst
--
diff --git a/doc/source/getting_started.rst b/doc/source/getting_started.rst
new file mode 100644
index 000..c30fb1e
--- /dev/null
+++ b/doc/source/getting_started.rst
@@ -0,0 +1,252 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. highlight:: none
+
+Getting Started
+===
+
+Installing Cassandra
+
+
+Prerequisites
+^
+
+- The latest version of Java 8, either the `Oracle Java Standard Edition 8
+  `__ or 
`OpenJDK 8 `__. To
+  verify that you have the correct version of java installed, type ``java 
-version``.
+
+- For using cqlsh, the latest version of `Python 2.7 
`__. To verify that you have
+  the correct version of Python installed, type ``python --version``.
+
+Installation from binary tarball files
+^^
+
+- Download the latest stable release from the `Apache Cassandra downloads 
website `__.
+
+- Untar the file somewhere, for example:
+
+::
+
+tar -xvf apache-cassandra-3.6-bin.tar.gz cassandra
+
+The files will be extracted into ``apache-cassandra-3.6``, you need to 
substitute 3.6 with the release number that you
+have downloaded.
+
+- Optionally add ``apache-cassandra-3.6\bin`` to your path.
+- Start Cassandra in the foreground by invoking ``bin/cassandra -f`` from the 
command line. Press "Control-C" to stop
+  Cassandra. Start Cassandra in the background by invoking ``bin/cassandra`` 
from the command line. Invoke ``kill pid``
+  or ``pkill -f CassandraDaemon`` to stop Cassandra, where pid is the 
Cassandra process id, which you can find for
+  example by invoking ``pgrep -f CassandraDaemon``.
+- Verify that Cassandra is running by invoking ``bin/nodetool status`` from 
the command line.
+- Configuration files are located in the ``conf`` sub-directory.
+- Since Cassandra 2.1, log and data directories are located in the ``logs`` 
and ``data`` sub-directories respectively.
+  Older versions defaulted to ``/var/log/cassandra`` and 
``/var/lib/cassandra``. Due to this, it is necessary to either
+  start Cassandra with root privileges or change ``conf/cassandra.yaml`` to 
use directories owned by the current user,
+  as explained below in the section on changing the location of directories.
+
+Installation from Debian packages
+^
+
+- Add the Apache repository of Cassandra to 
``/etc/apt/sources.list.d/cassandra.sources.list``, for example for version
+  3.6:
+
+::
+
+echo "deb http://www.apache.org/dist/cassandra/debian 36x main" | sudo tee 
-a /etc/apt/sources.list.d/cassandra.sources.list
+
+- Update the repositories:
+
+::
+
+sudo 

[07/34] cassandra git commit: Add initial version of security section

[slebresne]

Add initial version of security section


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/898b91a0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/898b91a0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/898b91a0

Branch: refs/heads/trunk
Commit: 898b91a029ff942b09221dea32d138ea988bd2ed
Parents: b1edbd1
Author: Sam Tunnicliffe 
Authored: Thu Jun 16 17:52:51 2016 +0100
Committer: Sylvain Lebresne 
Committed: Fri Jun 17 21:28:11 2016 +0200

--
 doc/source/operations.rst | 386 -
 1 file changed, 384 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/898b91a0/doc/source/operations.rst
--
diff --git a/doc/source/operations.rst b/doc/source/operations.rst
index 40b3e09..9e79700 100644
--- a/doc/source/operations.rst
+++ b/doc/source/operations.rst
@@ -437,15 +437,397 @@ Metric Reporters
 Security
 
 
+There are three main components to the security features provided by Cassandra:
+
+- TLS/SSL encryption for client and inter-node communication
+- Client authentication
+- Authorization
+
+TLS/SSL Encryption
+^^
+Cassandra provides secure communication between a client machine and a 
database cluster and between nodes within a
+cluster. Enabling encryption ensures that data in flight is not compromised 
and is transferred securely. The options for
+client-to-node and node-to-node encryption are managed separately and may be 
configured independently.
+
+In both cases, the JVM defaults for supported protocols and cipher suites are 
used when encryption is enabled. These can
+be overidden using the settings in ``cassandra.yaml``, but this is not 
recommended unless there are policies in place
+which dictate certain settings or a need to disable vulnerable ciphers or 
protocols in cases where the JVM cannot be
+updated.
+
+FIPS compliant settings can be configured at the JVM level and should not 
involve changing encryption settings in
+cassandra.yaml. See `the java document on FIPS 
`__
+for more details.
+
+For information on generating the keystore and truststore files used in SSL 
communications, see the
+`java documentation on creating keystores 
`__
+
+Inter-node Encryption
+~
+
+The settings for managing inter-node encryption are found in 
``cassandra.yaml`` in the ``server_encryption_options``
+section. To enable inter-node encryption, change the ``internode_encryption`` 
setting from its default value of ``none``
+to one value from: ``rack``, ``dc`` or ``all``.
+
+Client to Node Encryption
+~
+
+The settings for managing client to node encryption are found in 
``cassandra.yaml`` in the ``client_encryption_options``
+section. There are two primary toggles here for enabling encryption, 
``enabled`` and ``optional``.
+
+- If neither is set to ``true``, client connections are entirely unencrypted.
+- If ``enabled`` is set to ``true`` and ``optional`` is set to ``false``, all 
client connections must be secured.
+- If both options are set to ``true``, both encrypted and unencrypted 
connections are supported using the same port.
+  Client connections using encryption with this configuration will be 
automatically detected and handled by the server.
+
+As an alternative to the ``optional`` setting, separate ports can also be 
configured for secure and unsecure connections
+where operational requirements demand it. To do so, set ``optional`` to false 
and use the ``native_transport_port_ssl``
+setting in ``cassandra.yaml`` to specify the port to be used for secure client 
communication.
+
+.. _operation-roles:
+
 Roles
 ^
 
-.. todo:: todo
+Cassandra uses database roles, which may represent either a single user or a 
group of users, in both authentication and
+permissions management. Role management is an extension point in Cassandra and 
may be configured using the
+``role_manager`` setting in ``cassandra.yaml``. The default setting uses 
``CassandraRoleManager``, an implementation
+which stores role information in the tables of the ``system_auth`` keyspace.
+
+See also the :ref:`CQL documentation on roles `.
+
+Authentication
+^^
+
+Authentication is pluggable in Cassandra and is configured using the 
``authenticator`` setting in ``cassandra.yaml``.
+Cassandra ships with two options included in the default distribution.
+
+By default, Cassandra is configured with ``AllowAllAuthenticator`` which 
performs no authentication checks and therefore
+requires no credentials. It 

[16/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/tools/cqlsh.rst
--
diff --git a/doc/source/tools/cqlsh.rst b/doc/source/tools/cqlsh.rst
new file mode 100644
index 000..45e2db8
--- /dev/null
+++ b/doc/source/tools/cqlsh.rst
@@ -0,0 +1,455 @@
+.. highlight:: none
+
+.. _cqlsh:
+
+cqlsh: the CQL shell
+
+
+cqlsh is a command line shell for interacting with Cassandra through CQL (the 
Cassandra Query Language).  It is shipped
+with every Cassandra package, and can be found in the bin/ directory alongside 
the cassandra executable.  cqlsh utilizes
+the Python native protocol driver, and connects to the single node specified 
on the command line.
+
+
+Compatibility
+^
+
+cqlsh is compatible with Python 2.7.
+
+In general, a given version of cqlsh is only guaranteed to work with the 
version of Cassandra that it was released with.
+In some cases, cqlsh make work with older or newer versions of Cassandra, but 
this is not officially supported.
+
+
+Optional Dependencies
+^
+
+cqlsh ships with all essential dependencies.  However, there are some optional 
dependencies that can be installed to
+improve the capabilities of cqlsh.
+
+pytz
+
+
+By default, cqlsh displays all timestamps with a UTC timezone.  To support 
display of timestamps with another timezone,
+the `pytz `__ library must be installed.  See 
the ``timezone`` option in cqlshrc_ for
+specifying a timezone to use.
+
+cython
+~~
+
+The performance of cqlsh's ``COPY`` operations can be improved by installing 
`cython `__.  This will
+compile the python modules that are central to the performance of ``COPY``.
+
+cqlshrc
+^^^
+
+The ``cqlshrc`` file holds configuration options for cqlsh.  By default this 
is in the user's home directory at
+``~/.cassandra/cqlsh``, but a custom location can be specified with the 
``--cqlshrc`` option.
+
+Example config values and documentation can be found in the 
``conf/cqlshrc.sample`` file of a tarball installation.  You
+can also view the latest version of `cqlshrc online 
`__.
+
+
+Command Line Options
+
+
+Usage:
+
+``cqlsh [options] [host [port]]``
+
+Options:
+
+``-C`` ``--color``
+  Force color output
+
+``--no-color``
+  Disable color output
+
+``--browser``
+  Specify the browser to use for displaying cqlsh help.  This can be one of 
the `supported browser names
+  `__ (e.g. ``firefox``) or 
a browser path followed by ``%s`` (e.g.
+  ``/usr/bin/google-chrome-stable %s``).
+
+``--ssl``
+  Use SSL when connecting to Cassandra
+
+``-u`` ``--user``
+  Username to authenticate against Cassandra with
+
+``-p`` ``--password``
+  Password to authenticate against Cassandra with, should
+  be used in conjunction with ``--user``
+
+``-k`` ``--keyspace``
+  Keyspace to authenticate to, should be used in conjunction
+  with ``--user``
+
+``-f`` ``--file``
+  Execute commands from the given file, then exit
+
+``--debug``
+  Print additional debugging information
+
+``--encoding``
+  Specify a non-default encoding for output (defaults to UTF-8)
+
+``--cqlshrc``
+  Specify a non-default location for the ``cqlshrc`` file
+
+``-e`` ``--execute``
+  Execute the given statement, then exit
+
+``--connect-timeout``
+  Specify the connection timeout in seconds (defaults to 2s)
+
+``--request-timeout``
+  Specify the request timeout in seconds (defaults to 10s)
+
+``-t`` ``--tty``
+  Force tty mode (command prompt)
+
+
+Special Commands
+
+
+In addition to supporting regular CQL statements, cqlsh also supports a number 
of special commands that are not part of
+CQL.  These are detailed below.
+
+``CONSISTENCY``
+~~~
+
+`Usage`: ``CONSISTENCY ``
+
+Sets the consistency level for operations to follow.  Valid arguments include:
+
+- ``ANY``
+- ``ONE``
+- ``TWO``
+- ``THREE``
+- ``QUORUM``
+- ``ALL``
+- ``LOCAL_QUORUM``
+- ``LOCAL_ONE``
+- ``SERIAL``
+- ``LOCAL_SERIAL``
+
+``SERIAL CONSISTENCY``
+~~
+
+`Usage`: ``SERIAL CONSISTENCY ``
+
+Sets the serial consistency level for operations to follow.  Valid arguments 
include:
+
+- ``SERIAL``
+- ``LOCAL_SERIAL``
+
+The serial consistency level is only used by conditional updates (``INSERT``, 
``UPDATE`` and ``DELETE`` with an ``IF``
+condition). For those, the serial consistency level defines the consistency 
level of the serial phase (or “paxos” phase)
+while the normal consistency level defines the consistency for the “learn” 
phase, i.e. what type of reads will be
+guaranteed to see the update right away. For example, if a conditional write 
has a consistency level of ``QUORUM`` (and
+is successful), then a ``QUORUM`` read is guaranteed to see that write. But if 
the regular consistency level 

[24/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/configuration/cassandra_config_file.rst
--
diff --git a/doc/source/configuration/cassandra_config_file.rst 
b/doc/source/configuration/cassandra_config_file.rst
new file mode 100644
index 000..b7d1bbc
--- /dev/null
+++ b/doc/source/configuration/cassandra_config_file.rst
@@ -0,0 +1,1699 @@
+Cassandra Configuration File
+=
+
+``cluster_name``
+
+The name of the cluster. This is mainly used to prevent machines in
+one logical cluster from joining another.
+
+*Default Value:* 'Test Cluster'
+
+``num_tokens``
+--
+
+This defines the number of tokens randomly assigned to this node on the ring
+The more tokens, relative to other nodes, the larger the proportion of data
+that this node will store. You probably want all nodes to have the same number
+of tokens assuming they have equal hardware capability.
+
+If you leave this unspecified, Cassandra will use the default of 1 token for 
legacy compatibility,
+and will use the initial_token as described below.
+
+Specifying initial_token will override this setting on the node's initial 
start,
+on subsequent starts, this setting will apply even if initial token is set.
+
+If you already have a cluster with 1 token per node, and wish to migrate to 
+multiple tokens per node, see http://wiki.apache.org/cassandra/Operations
+
+*Default Value:* 256
+
+``allocate_tokens_for_keyspace``
+
+*This option is commented out by default.*
+
+Triggers automatic allocation of num_tokens tokens for this node. The 
allocation
+algorithm attempts to choose tokens in a way that optimizes replicated load 
over
+the nodes in the datacenter for the replication strategy used by the specified
+keyspace.
+
+The load assigned to each node will be close to proportional to its number of
+vnodes.
+
+Only supported with the Murmur3Partitioner.
+
+*Default Value:* KEYSPACE
+
+``initial_token``
+-
+*This option is commented out by default.*
+
+initial_token allows you to specify tokens manually.  While you can use it with
+vnodes (num_tokens > 1, above) -- in which case you should provide a 
+comma-separated list -- it's primarily used when adding nodes to legacy 
clusters 
+that do not have vnodes enabled.
+
+``hinted_handoff_enabled``
+--
+
+See http://wiki.apache.org/cassandra/HintedHandoff
+May either be "true" or "false" to enable globally
+
+*Default Value:* true
+
+``hinted_handoff_disabled_datacenters``
+---
+*This option is commented out by default.*
+
+When hinted_handoff_enabled is true, a black list of data centers that will not
+perform hinted handoff
+
+*Default Value (complex option)*::
+
+#- DC1
+#- DC2
+
+``max_hint_window_in_ms``
+-
+this defines the maximum amount of time a dead host will have hints
+generated.  After it has been dead this long, new hints for it will not be
+created until it has been seen alive and gone down again.
+
+*Default Value:* 1080 # 3 hours
+
+``hinted_handoff_throttle_in_kb``
+-
+
+Maximum throttle in KBs per second, per delivery thread.  This will be
+reduced proportionally to the number of nodes in the cluster.  (If there
+are two nodes in the cluster, each delivery thread will use the maximum
+rate; if there are three, each will throttle to half of the maximum,
+since we expect two nodes to be delivering hints simultaneously.)
+
+*Default Value:* 1024
+
+``max_hints_delivery_threads``
+--
+
+Number of threads with which to deliver hints;
+Consider increasing this number when you have multi-dc deployments, since
+cross-dc handoff tends to be slower
+
+*Default Value:* 2
+
+``hints_directory``
+---
+*This option is commented out by default.*
+
+Directory where Cassandra should store hints.
+If not set, the default directory is $CASSANDRA_HOME/data/hints.
+
+*Default Value:*  /var/lib/cassandra/hints
+
+``hints_flush_period_in_ms``
+
+
+How often hints should be flushed from the internal buffers to disk.
+Will *not* trigger fsync.
+
+*Default Value:* 1
+
+``max_hints_file_size_in_mb``
+-
+
+Maximum size for a single hints file, in megabytes.
+
+*Default Value:* 128
+
+``hints_compression``
+-
+*This option is commented out by default.*
+
+Compression to apply to the hint files. If omitted, hints files
+will be written uncompressed. LZ4, Snappy, and Deflate compressors
+are supported.
+
+*Default Value (complex option)*::
+
+#   - class_name: LZ4Compressor
+# parameters:
+# -
+
+``batchlog_replay_throttle_in_kb``
+--
+Maximum throttle in KBs per second, total. This will be
+reduced proportionally to the numb

[20/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/cql/security.rst
--
diff --git a/doc/source/cql/security.rst b/doc/source/cql/security.rst
new file mode 100644
index 000..f119c22
--- /dev/null
+++ b/doc/source/cql/security.rst
@@ -0,0 +1,637 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. highlight:: sql
+
+.. _cql-security:
+
+Security
+
+
+.. _roles:
+
+Database Roles
+^^
+
+CREATE ROLE
+~~~
+
+*Syntax:*
+
+| bc(syntax)..
+|  ::= CREATE ROLE ( IF NOT EXISTS )? ( WITH ( AND )\* )?
+
+|  ::= PASSWORD = 
+|  \| LOGIN = 
+|  \| SUPERUSER = 
+|  \| OPTIONS = 
+| p.
+
+*Sample:*
+
+| bc(sample).
+| CREATE ROLE new\_role;
+| CREATE ROLE alice WITH PASSWORD = ‘password\_a’ AND LOGIN = true;
+| CREATE ROLE bob WITH PASSWORD = ‘password\_b’ AND LOGIN = true AND
+  SUPERUSER = true;
+| CREATE ROLE carlos WITH OPTIONS = { ‘custom\_option1’ :
+  ‘option1\_value’, ‘custom\_option2’ : 99 };
+
+By default roles do not possess ``LOGIN`` privileges or ``SUPERUSER``
+status.
+
+`Permissions <#permissions>`__ on database resources are granted to
+roles; types of resources include keyspaces, tables, functions and roles
+themselves. Roles may be granted to other roles to create hierarchical
+permissions structures; in these hierarchies, permissions and
+``SUPERUSER`` status are inherited, but the ``LOGIN`` privilege is not.
+
+If a role has the ``LOGIN`` privilege, clients may identify as that role
+when connecting. For the duration of that connection, the client will
+acquire any roles and privileges granted to that role.
+
+Only a client with with the ``CREATE`` permission on the database roles
+resource may issue ``CREATE ROLE`` requests (see the `relevant
+section <#permissions>`__ below), unless the client is a ``SUPERUSER``.
+Role management in Cassandra is pluggable and custom implementations may
+support only a subset of the listed options.
+
+Role names should be quoted if they contain non-alphanumeric characters.
+
+.. _setting-credentials-for-internal-authentication:
+
+Setting credentials for internal authentication
+```
+
+| Use the ``WITH PASSWORD`` clause to set a password for internal
+  authentication, enclosing the password in single quotation marks.
+| If internal authentication has not been set up or the role does not
+  have ``LOGIN`` privileges, the ``WITH PASSWORD`` clause is not
+  necessary.
+
+Creating a role conditionally
+`
+
+Attempting to create an existing role results in an invalid query
+condition unless the ``IF NOT EXISTS`` option is used. If the option is
+used and the role exists, the statement is a no-op.
+
+| bc(sample).
+| CREATE ROLE other\_role;
+| CREATE ROLE IF NOT EXISTS other\_role;
+
+ALTER ROLE
+~~
+
+*Syntax:*
+
+| bc(syntax)..
+|  ::= ALTER ROLE ( WITH ( AND )\* )?
+
+|  ::= PASSWORD = 
+|  \| LOGIN = 
+|  \| SUPERUSER = 
+|  \| OPTIONS = 
+| p.
+
+*Sample:*
+
+| bc(sample).
+| ALTER ROLE bob WITH PASSWORD = ‘PASSWORD\_B’ AND SUPERUSER = false;
+
+Conditions on executing ``ALTER ROLE`` statements:
+
+-  A client must have ``SUPERUSER`` status to alter the ``SUPERUSER``
+   status of another role
+-  A client cannot alter the ``SUPERUSER`` status of any role it
+   currently holds
+-  A client can only modify certain properties of the role with which it
+   identified at login (e.g. ``PASSWORD``)
+-  To modify properties of a role, the client must be granted ``ALTER``
+   `permission <#permissions>`__ on that role
+
+DROP ROLE
+~
+
+*Syntax:*
+
+| bc(syntax)..
+|  ::= DROP ROLE ( IF EXISTS )? 
+| p.
+
+*Sample:*
+
+| bc(sample).
+| DROP ROLE alice;
+| DROP ROLE IF EXISTS bob;
+
+| ``DROP ROLE`` requires the client to have ``DROP``
+  `permission <#permissions>`__ on the role in question. In addition,
+  client may not ``DROP`` the role with which it identified at login.
+  Finaly, only a client with ``SUPERUSER`` status may ``DROP`` another
+  ``SUPERUSER`` role.
+| Attempting to drop a role which does not exist results in an invalid
+  query condition unless the ``IF EXISTS`` option 

[27/34] cassandra git commit: Finish fixing the CQL doc

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e624238/doc/source/cql/security.rst
--
diff --git a/doc/source/cql/security.rst b/doc/source/cql/security.rst
index f119c22..aa65383 100644
--- a/doc/source/cql/security.rst
+++ b/doc/source/cql/security.rst
@@ -21,53 +21,47 @@
 Security
 
 
-.. _roles:
+.. _cql-roles:
 
 Database Roles
 ^^
 
+.. _create-role-statement:
+
 CREATE ROLE
 ~~~
 
-*Syntax:*
-
-| bc(syntax)..
-|  ::= CREATE ROLE ( IF NOT EXISTS )? ( WITH ( AND )\* )?
+Creating a role uses the ``CREATE ROLE`` statement:
 
-|  ::= PASSWORD = 
-|  \| LOGIN = 
-|  \| SUPERUSER = 
-|  \| OPTIONS = 
-| p.
+.. productionlist::
+   create_role_statement: CREATE ROLE [ IF NOT EXISTS ] `role_name`
+: [ WITH `role_options` ]
+   role_options: `role_option` ( AND `role_option` )*
+   role_option: PASSWORD '=' `string`
+  :| LOGIN '=' `boolean`
+  :| SUPERUSER '=' `boolean`
+  :| OPTIONS '=' `map_literal`
 
-*Sample:*
+For instance::
 
-| bc(sample).
-| CREATE ROLE new\_role;
-| CREATE ROLE alice WITH PASSWORD = ‘password\_a’ AND LOGIN = true;
-| CREATE ROLE bob WITH PASSWORD = ‘password\_b’ AND LOGIN = true AND
-  SUPERUSER = true;
-| CREATE ROLE carlos WITH OPTIONS = { ‘custom\_option1’ :
-  ‘option1\_value’, ‘custom\_option2’ : 99 };
+CREATE ROLE new_role;
+CREATE ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
+CREATE ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND 
SUPERUSER = true;
+CREATE ROLE carlos WITH OPTIONS = { 'custom_option1' : 'option1_value', 
'custom_option2' : 99 };
 
-By default roles do not possess ``LOGIN`` privileges or ``SUPERUSER``
-status.
+By default roles do not possess ``LOGIN`` privileges or ``SUPERUSER`` status.
 
-`Permissions <#permissions>`__ on database resources are granted to
-roles; types of resources include keyspaces, tables, functions and roles
-themselves. Roles may be granted to other roles to create hierarchical
-permissions structures; in these hierarchies, permissions and
-``SUPERUSER`` status are inherited, but the ``LOGIN`` privilege is not.
+:ref:`Permissions ` on database resources are granted to 
roles; types of resources include keyspaces,
+tables, functions and roles themselves. Roles may be granted to other roles to 
create hierarchical permissions
+structures; in these hierarchies, permissions and ``SUPERUSER`` status are 
inherited, but the ``LOGIN`` privilege is
+not.
 
-If a role has the ``LOGIN`` privilege, clients may identify as that role
-when connecting. For the duration of that connection, the client will
-acquire any roles and privileges granted to that role.
+If a role has the ``LOGIN`` privilege, clients may identify as that role when 
connecting. For the duration of that
+connection, the client will acquire any roles and privileges granted to that 
role.
 
-Only a client with with the ``CREATE`` permission on the database roles
-resource may issue ``CREATE ROLE`` requests (see the `relevant
-section <#permissions>`__ below), unless the client is a ``SUPERUSER``.
-Role management in Cassandra is pluggable and custom implementations may
-support only a subset of the listed options.
+Only a client with with the ``CREATE`` permission on the database roles 
resource may issue ``CREATE ROLE`` requests (see
+the :ref:`relevant section ` below), unless the client is a 
``SUPERUSER``. Role management in Cassandra
+is pluggable and custom implementations may support only a subset of the 
listed options.
 
 Role names should be quoted if they contain non-alphanumeric characters.
 
@@ -76,562 +70,428 @@ Role names should be quoted if they contain 
non-alphanumeric characters.
 Setting credentials for internal authentication
 ```
 
-| Use the ``WITH PASSWORD`` clause to set a password for internal
-  authentication, enclosing the password in single quotation marks.
-| If internal authentication has not been set up or the role does not
-  have ``LOGIN`` privileges, the ``WITH PASSWORD`` clause is not
-  necessary.
+Use the ``WITH PASSWORD`` clause to set a password for internal 
authentication, enclosing the password in single
+quotation marks.
+
+If internal authentication has not been set up or the role does not have 
``LOGIN`` privileges, the ``WITH PASSWORD``
+clause is not necessary.
 
 Creating a role conditionally
 `
 
-Attempting to create an existing role results in an invalid query
-condition unless the ``IF NOT EXISTS`` option is used. If the option is
-used and the role exists, the statement is a no-op.
+Attempting to create an existing role results in an invalid query condition 
unless the ``IF NOT EXISTS`` option is used.
+If the option is used and the role exists, the statement is a no-op::
+
+CREATE ROLE other_role;
+CREATE ROLE IF NOT EXISTS o

[23/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/cql.rst
--
diff --git a/doc/source/cql.rst b/doc/source/cql.rst
deleted file mode 100644
index 8d185a2..000
--- a/doc/source/cql.rst
+++ /dev/null
@@ -1,4114 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
-.. or more contributor license agreements.  See the NOTICE file
-.. distributed with this work for additional information
-.. regarding copyright ownership.  The ASF licenses this file
-.. to you under the Apache License, Version 2.0 (the
-.. "License"); you may not use this file except in compliance
-.. with the License.  You may obtain a copy of the License at
-..
-.. http://www.apache.org/licenses/LICENSE-2.0
-..
-.. Unless required by applicable law or agreed to in writing, software
-.. distributed under the License is distributed on an "AS IS" BASIS,
-.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-.. See the License for the specific language governing permissions and
-.. limitations under the License.
-
-.. highlight:: sql
-
-.. _UUID: https://en.wikipedia.org/wiki/Universally_unique_identifier
-
-The Cassandra Query Language (CQL)
-==
-
-This document describes the Cassandra Query Language (CQL) [#]_. Note that 
this document describes the last version of
-the languages. However, the `changes <#changes>`_ section provides the diff 
between the different versions of CQL.
-
-CQL offers a model close to SQL in the sense that data is put in *tables* 
containing *rows* of *columns*. For
-that reason, when used in this document, these terms (tables, rows and 
columns) have the same definition than they have
-in SQL. But please note that as such, they do **not** refer to the concept of 
rows and columns found in the deprecated
-thrift API (and earlier version 1 and 2 of CQL).
-
-.. _definitions:
-
-Definitions

-
-.. _conventions:
-
-Conventions
-^^^
-
-To aid in specifying the CQL syntax, we will use the following conventions in 
this document:
-
-- Language rules will be given in an informal `BNF variant
-  `_ notation. 
In particular, we'll use square brakets
-  (``[ item ]``) for optional items, ``*`` and ``+`` for repeated items (where 
``+`` imply at least one).
-- The grammar will also use the following convention for convenience: 
non-terminal term will be lowercase (and link to
-  their definition) while terminal keywords will be provided "all caps". Note 
however that keywords are
-  :ref:`identifiers` and are thus case insensitive in practice. We will also 
define some early construction using
-  regexp, which we'll indicate with ``re()``.
-- The grammar is provided for documentation purposes and leave some minor 
details out.  For instance, the comma on the
-  last column definition in a ``CREATE TABLE`` statement is optional but 
supported if present even though the grammar in
-  this document suggests otherwise. Also, not everything accepted by the 
grammar is necessarily valid CQL.
-- References to keywords or pieces of CQL code in running text will be shown 
in a ``fixed-width font``.
-
-
-.. _identifiers:
-
-Identifiers and keywords
-
-
-The CQL language uses *identifiers* (or *names*) to identify tables, columns 
and other objects. An identifier is a token
-matching the regular expression ``[a-zA-Z][a-zA-Z0-9_]*``.
-
-A number of such identifiers, like ``SELECT`` or ``WITH``, are *keywords*. 
They have a fixed meaning for the language
-and most are reserved. The list of those keywords can be found in 
:ref:`appendix-A`.
-
-Identifiers and (unquoted) keywords are case insensitive. Thus ``SELECT`` is 
the same than ``select`` or ``sElEcT``, and
-``myId`` is the same than ``myid`` or ``MYID``. A convention often used (in 
particular by the samples of this
-documentation) is to use upper case for keywords and lower case for other 
identifiers.
-
-There is a second kind of identifiers called *quoted identifiers* defined by 
enclosing an arbitrary sequence of
-characters (non empty) in double-quotes(``"``). Quoted identifiers are never 
keywords. Thus ``"select"`` is not a
-reserved keyword and can be used to refer to a column (note that using this is 
particularly advised), while ``select``
-would raise a parsing error. Also, contrarily to unquoted identifiers and 
keywords, quoted identifiers are case
-sensitive (``"My Quoted Id"`` is *different* from ``"my quoted id"``). A fully 
lowercase quoted identifier that matches
-``[a-zA-Z][a-zA-Z0-9_]*`` is however *equivalent* to the unquoted identifier 
obtained by removing the double-quote (so
-``"myid"`` is equivalent to ``myid`` and to ``myId`` but different from 
``"myId"``).  Inside a quoted identifier, the
-double-quote character can be repeated to escape it, so ``"foo "" bar"`` is a 
valid identifier.

[15/34] cassandra git commit: Automatically generate docs for cassandra.yaml

[slebresne]

Automatically generate docs for cassandra.yaml


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2f30714
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2f30714
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2f30714

Branch: refs/heads/trunk
Commit: f2f30714436457dcb175b6365bf85d116f3763d9
Parents: 5d65542
Author: Tyler Hobbs 
Authored: Tue Jun 21 12:40:40 2016 -0500
Committer: Sylvain Lebresne 
Committed: Tue Jun 21 19:53:48 2016 +0200

--
 conf/cassandra.yaml| 222 +---
 doc/Makefile   |  27 +
 doc/convert_yaml_to_rst.py | 144 ++
 doc/source/index.rst   |   1 +
 4 files changed, 314 insertions(+), 80 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f2f30714/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index dcd5278..c43820e 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -35,20 +35,22 @@ num_tokens: 256
 # Only supported with the Murmur3Partitioner.
 # allocate_tokens_for_keyspace: KEYSPACE
 
-# initial_token allows you to specify tokens manually.  While you can use # it 
with
+# initial_token allows you to specify tokens manually.  While you can use it 
with
 # vnodes (num_tokens > 1, above) -- in which case you should provide a 
-# comma-separated list -- it's primarily used when adding nodes # to legacy 
clusters 
+# comma-separated list -- it's primarily used when adding nodes to legacy 
clusters 
 # that do not have vnodes enabled.
 # initial_token:
 
 # See http://wiki.apache.org/cassandra/HintedHandoff
 # May either be "true" or "false" to enable globally
 hinted_handoff_enabled: true
+
 # When hinted_handoff_enabled is true, a black list of data centers that will 
not
 # perform hinted handoff
-#hinted_handoff_disabled_datacenters:
+# hinted_handoff_disabled_datacenters:
 #- DC1
 #- DC2
+
 # this defines the maximum amount of time a dead host will have hints
 # generated.  After it has been dead this long, new hints for it will not be
 # created until it has been seen alive and gone down again.
@@ -193,26 +195,44 @@ partitioner: org.apache.cassandra.dht.Murmur3Partitioner
 # If not set, the default directory is $CASSANDRA_HOME/data/commitlog.
 # commitlog_directory: /var/lib/cassandra/commitlog
 
-# policy for data disk failures:
-# die: shut down gossip and client transports and kill the JVM for any fs 
errors or
-#  single-sstable errors, so the node can be replaced.
-# stop_paranoid: shut down gossip and client transports even for 
single-sstable errors,
-#kill the JVM for errors during startup.
-# stop: shut down gossip and client transports, leaving the node effectively 
dead, but
-#   can still be inspected via JMX, kill the JVM for errors during startup.
-# best_effort: stop using the failed disk and respond to requests based on
-#  remaining available sstables.  This means you WILL see obsolete
-#  data at CL.ONE!
-# ignore: ignore fatal errors and let requests fail, as in pre-1.2 Cassandra
+# Policy for data disk failures:
+#
+# die
+#   shut down gossip and client transports and kill the JVM for any fs errors 
or
+#   single-sstable errors, so the node can be replaced.
+#
+# stop_paranoid
+#   shut down gossip and client transports even for single-sstable errors,
+#   kill the JVM for errors during startup.
+#
+# stop
+#   shut down gossip and client transports, leaving the node effectively dead, 
but
+#   can still be inspected via JMX, kill the JVM for errors during startup.
+#
+# best_effort
+#stop using the failed disk and respond to requests based on
+#remaining available sstables.  This means you WILL see obsolete
+#data at CL.ONE!
+#
+# ignore
+#ignore fatal errors and let requests fail, as in pre-1.2 Cassandra
 disk_failure_policy: stop
 
-# policy for commit disk failures:
-# die: shut down gossip and Thrift and kill the JVM, so the node can be 
replaced.
-# stop: shut down gossip and Thrift, leaving the node effectively dead, but
-#   can still be inspected via JMX.
-# stop_commit: shutdown the commit log, letting writes collect but
-#  continuing to service reads, as in pre-2.0.5 Cassandra
-# ignore: ignore fatal errors and let the batches fail
+# Policy for commit disk failures:
+#
+# die
+#   shut down gossip and Thrift and kill the JVM, so the node can be replaced.
+#
+# stop
+#   shut down gossip and Thrift, leaving the node effectively dead, but
+#   can still be inspected via JMX.
+#
+# stop_commit
+#   shutdown the commit log, letting writes collect but
+#   continuing to service reads, as in pre-2.0.5 Cassandra
+#
+# ignor

[17/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/operations.rst
--
diff --git a/doc/source/operations.rst b/doc/source/operations.rst
deleted file mode 100644
index cf8812f..000
--- a/doc/source/operations.rst
+++ /dev/null
@@ -1,1900 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
-.. or more contributor license agreements.  See the NOTICE file
-.. distributed with this work for additional information
-.. regarding copyright ownership.  The ASF licenses this file
-.. to you under the Apache License, Version 2.0 (the
-.. "License"); you may not use this file except in compliance
-.. with the License.  You may obtain a copy of the License at
-..
-.. http://www.apache.org/licenses/LICENSE-2.0
-..
-.. Unless required by applicable law or agreed to in writing, software
-.. distributed under the License is distributed on an "AS IS" BASIS,
-.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-.. See the License for the specific language governing permissions and
-.. limitations under the License.
-
-.. highlight:: none
-
-Operating Cassandra
-===
-
-Snitch
---
-
-In cassandra, the snitch has two functions:
-
-- it teaches Cassandra enough about your network topology to route requests 
efficiently.
-- it allows Cassandra to spread replicas around your cluster to avoid 
correlated failures. It does this by grouping
-  machines into "datacenters" and "racks."  Cassandra will do its best not to 
have more than one replica on the same
-  "rack" (which may not actually be a physical location).
-
-Dynamic snitching
-^
-
-The dynamic snitch monitor read latencies to avoid reading from hosts that 
have slowed down. The dynamic snitch is
-configured with the following properties on ``cassandra.yaml``:
-
-- ``dynamic_snitch``: whether the dynamic snitch should be enabled or disabled.
-- ``dynamic_snitch_update_interval_in_ms``: controls how often to perform the 
more expensive part of host score
-  calculation.
-- ``dynamic_snitch_reset_interval_in_ms``: if set greater than zero and 
read_repair_chance is < 1.0, this will allow
-  'pinning' of replicas to hosts in order to increase cache capacity.
-- ``dynamic_snitch_badness_threshold:``: The badness threshold will control 
how much worse the pinned host has to be
-  before the dynamic snitch will prefer other replicas over it.  This is 
expressed as a double which represents a
-  percentage.  Thus, a value of 0.2 means Cassandra would continue to prefer 
the static snitch values until the pinned
-  host was 20% worse than the fastest.
-
-Snitch classes
-^^
-
-The ``endpoint_snitch`` parameter in ``cassandra.yaml`` should be set to the 
class the class that implements
-``IEndPointSnitch`` which will be wrapped by the dynamic snitch and decide if 
two endpoints are in the same data center
-or on the same rack. Out of the box, Cassandra provides the snitch 
implementations:
-
-GossipingPropertyFileSnitch
-This should be your go-to snitch for production use. The rack and 
datacenter for the local node are defined in
-cassandra-rackdc.properties and propagated to other nodes via gossip. If 
``cassandra-topology.properties`` exists,
-it is used as a fallback, allowing migration from the PropertyFileSnitch.
-
-SimpleSnitch
-Treats Strategy order as proximity. This can improve cache locality when 
disabling read repair. Only appropriate for
-single-datacenter deployments.
-
-PropertyFileSnitch
-Proximity is determined by rack and data center, which are explicitly 
configured in
-``cassandra-topology.properties``.
-
-Ec2Snitch
-Appropriate for EC2 deployments in a single Region. Loads Region and 
Availability Zone information from the EC2 API.
-The Region is treated as the datacenter, and the Availability Zone as the 
rack. Only private IPs are used, so this
-will not work across multiple regions.
-
-Ec2MultiRegionSnitch
-Uses public IPs as broadcast_address to allow cross-region connectivity 
(thus, you should set seed addresses to the
-public IP as well). You will need to open the ``storage_port`` or 
``ssl_storage_port`` on the public IP firewall
-(For intra-Region traffic, Cassandra will switch to the private IP after 
establishing a connection).
-
-RackInferringSnitch
-Proximity is determined by rack and data center, which are assumed to 
correspond to the 3rd and 2nd octet of each
-node's IP address, respectively.  Unless this happens to match your 
deployment conventions, this is best used as an
-example of writing a custom Snitch class and is provided in that spirit.
-
-Adding, replacing, moving and removing nodes
-
-
-Bootstrap
-^
-
-Adding new nodes is called "bootstrapping". The ``num_tokens`` parameter will 
define the amount of virtual nodes
-(tokens) the joinin

[14/34] cassandra git commit: Fix CQL doc (incomplete)

[slebresne]

Fix CQL doc (incomplete)


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d65542b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d65542b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d65542b

Branch: refs/heads/trunk
Commit: 5d65542bd3b5cbdfe8ebed7892fce7e06e7779f0
Parents: 0ae8485
Author: Sylvain Lebresne 
Authored: Wed Jun 15 09:29:22 2016 +0200
Committer: Sylvain Lebresne 
Committed: Tue Jun 21 16:38:24 2016 +0200

--
 doc/cql3/CQL.textile |4 +-
 doc/source/_static/extra.css |4 +
 doc/source/architecture.rst  |2 +
 doc/source/cql.rst   | 3708 +++--
 doc/source/operations.rst|   15 +-
 5 files changed, 1967 insertions(+), 1766 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d65542b/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 275717c..e354524 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -1678,7 +1678,9 @@ Show any permissions granted to @carlos@ or any of 
@carlos@'s roles, limited to
 
 h2(#types). Data Types
 
-CQL supports a rich set of data types for columns defined in a table, 
including collection types. On top of those native and collection types, users 
can also provide custom types (through a JAVA class extending @AbstractType@ 
loadable by Cassandra). The syntax of types is thus:
+CQL supports a rich set of data types for columns defined in a table, 
including collection types. On top of those native
+and collection types, users can also provide custom types (through a JAVA 
class extending @AbstractType@ loadable by
+Cassandra). The syntax of types is thus:
 
 bc(syntax).. 
  ::= 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d65542b/doc/source/_static/extra.css
--
diff --git a/doc/source/_static/extra.css b/doc/source/_static/extra.css
index ff9f1d1..1b65a86 100644
--- a/doc/source/_static/extra.css
+++ b/doc/source/_static/extra.css
@@ -12,10 +12,14 @@ a.reference.internal code.literal {
 border: none;
 font-size: 12px;
 color: #2980B9;
+padding: 0;
+background: none;
 }
 
 a.reference.internal:visited code.literal {
 color: #9B59B6;
+padding: 0;
+background: none;
 }
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d65542b/doc/source/architecture.rst
--
diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst
index cb52477..9209414 100644
--- a/doc/source/architecture.rst
+++ b/doc/source/architecture.rst
@@ -40,6 +40,8 @@ Token Ring/Ranges
 
 .. todo:: todo
 
+.. _replication-strategy:
+
 Replication
 ^^^
 

[04/34] cassandra git commit: Add snitch and range movements section on Operations

[slebresne]

Add snitch and range movements section on Operations


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4d444022
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4d444022
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4d444022

Branch: refs/heads/trunk
Commit: 4d444022c3ace5928a15b2a54af0f95f4191c0d7
Parents: cad277b
Author: Paulo Motta 
Authored: Tue Jun 14 17:54:38 2016 -0300
Committer: Sylvain Lebresne 
Committed: Wed Jun 15 10:31:17 2016 +0200

--
 doc/source/operations.rst | 164 -
 1 file changed, 160 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d444022/doc/source/operations.rst
--
diff --git a/doc/source/operations.rst b/doc/source/operations.rst
index 8228746..40b3e09 100644
--- a/doc/source/operations.rst
+++ b/doc/source/operations.rst
@@ -24,15 +24,171 @@ Replication Strategies
 
 .. todo:: todo
 
-Snitches
-
+Snitch
+--
 
-.. todo:: todo
+In cassandra, the snitch has two functions:
+
+- it teaches Cassandra enough about your network topology to route requests 
efficiently.
+- it allows Cassandra to spread replicas around your cluster to avoid 
correlated failures. It does this by grouping
+  machines into "datacenters" and "racks."  Cassandra will do its best not to 
have more than one replica on the same
+  "rack" (which may not actually be a physical location).
+
+Dynamic snitching
+^
+
+The dynamic snitch monitor read latencies to avoid reading from hosts that 
have slowed down. The dynamic snitch is
+configured with the following properties on ``cassandra.yaml``:
+
+- ``dynamic_snitch``: whether the dynamic snitch should be enabled or disabled.
+- ``dynamic_snitch_update_interval_in_ms``: controls how often to perform the 
more expensive part of host score
+  calculation.
+- ``dynamic_snitch_reset_interval_in_ms``: if set greater than zero and 
read_repair_chance is < 1.0, this will allow
+  'pinning' of replicas to hosts in order to increase cache capacity.
+- ``dynamic_snitch_badness_threshold:``: The badness threshold will control 
how much worse the pinned host has to be
+  before the dynamic snitch will prefer other replicas over it.  This is 
expressed as a double which represents a
+  percentage.  Thus, a value of 0.2 means Cassandra would continue to prefer 
the static snitch values until the pinned
+  host was 20% worse than the fastest.
+
+Snitch classes
+^^
+
+The ``endpoint_snitch`` parameter in ``cassandra.yaml`` should be set to the 
class the class that implements
+``IEndPointSnitch`` which will be wrapped by the dynamic snitch and decide if 
two endpoints are in the same data center
+or on the same rack. Out of the box, Cassandra provides the snitch 
implementations:
+
+GossipingPropertyFileSnitch
+This should be your go-to snitch for production use. The rack and 
datacenter for the local node are defined in
+cassandra-rackdc.properties and propagated to other nodes via gossip. If 
``cassandra-topology.properties`` exists,
+it is used as a fallback, allowing migration from the PropertyFileSnitch.
+
+SimpleSnitch
+Treats Strategy order as proximity. This can improve cache locality when 
disabling read repair. Only appropriate for
+single-datacenter deployments.
+
+PropertyFileSnitch
+Proximity is determined by rack and data center, which are explicitly 
configured in
+``cassandra-topology.properties``.
+
+Ec2Snitch
+Appropriate for EC2 deployments in a single Region. Loads Region and 
Availability Zone information from the EC2 API.
+The Region is treated as the datacenter, and the Availability Zone as the 
rack. Only private IPs are used, so this
+will not work across multiple regions.
+
+Ec2MultiRegionSnitch
+Uses public IPs as broadcast_address to allow cross-region connectivity 
(thus, you should set seed addresses to the
+public IP as well). You will need to open the ``storage_port`` or 
``ssl_storage_port`` on the public IP firewall
+(For intra-Region traffic, Cassandra will switch to the private IP after 
establishing a connection).
+
+RackInferringSnitch
+Proximity is determined by rack and data center, which are assumed to 
correspond to the 3rd and 2nd octet of each
+node's IP address, respectively.  Unless this happens to match your 
deployment conventions, this is best used as an
+example of writing a custom Snitch class and is provided in that spirit.
 
 Adding, replacing, moving and removing nodes
 
 
-.. todo:: todo
+Bootstrap
+^
+
+Adding new nodes is called "bootstrapping". The ``num_tokens`` parameter will 
define the amoun

[11/34] cassandra git commit: Add Change Data Capture documentation

[slebresne]

Add Change Data Capture documentation


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/51b939c9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/51b939c9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/51b939c9

Branch: refs/heads/trunk
Commit: 51b939c91db5d1a7664d76c8f57160f2570ee1dd
Parents: 7bf837c
Author: Josh McKenzie 
Authored: Mon Jun 20 13:38:00 2016 -0400
Committer: Sylvain Lebresne 
Committed: Tue Jun 21 14:12:59 2016 +0200

--
 doc/source/operations.rst | 75 --
 1 file changed, 73 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/51b939c9/doc/source/operations.rst
--
diff --git a/doc/source/operations.rst b/doc/source/operations.rst
index 9094766..d7fcafb 100644
--- a/doc/source/operations.rst
+++ b/doc/source/operations.rst
@@ -338,7 +338,6 @@ There is a number of common options for all the compaction 
strategies;
 ``enabled`` (default: true)
 Whether minor compactions should run. Note that you can have 'enabled': 
true as a compaction option and then do
 'nodetool enableautocompaction' to start running compactions.
-Default true.
 ``tombstone_threshold`` (default: 0.2)
 How much of the sstable should be tombstones for us to consider doing a 
single sstable compaction of that sstable.
 ``tombstone_compaction_interval`` (default: 86400s (1 day))
@@ -738,7 +737,7 @@ similar text columns (such as repeated JSON blobs) often 
compress very well.
 Operational Impact
 ^^
 
-- Compression metadata is stored offheap and scales with data on disk.  This 
often requires 1-3GB of offheap RAM per
+- Compression metadata is stored off-heap and scales with data on disk.  This 
often requires 1-3GB of off-heap RAM per
   terabyte of data on disk, though the exact usage varies with 
``chunk_length_in_kb`` and compression ratios.
 
 - Streaming operations involve compressing and decompressing data on 
compressed tables - in some code paths (such as
@@ -754,6 +753,78 @@ Advanced Use
 Advanced users can provide their own compression class by implementing the 
interface at
 ``org.apache.cassandra.io.compress.ICompressor``.
 
+Change Data Capture
+---
+
+Overview
+
+
+Change data capture (CDC) provides a mechanism to flag specific tables for 
archival as well as rejecting writes to those
+tables once a configurable size-on-disk for the combined flushed and unflushed 
CDC-log is reached. An operator can
+enable CDC on a table by setting the table property ``cdc=true`` (either when 
:ref:`creating the table
+` or :ref:`altering it `), 
after which any CommitLogSegments containing
+data for a CDC-enabled table are moved to the directory specified in 
``cassandra.yaml`` on segment discard. A threshold
+of total disk space allowed is specified in the yaml at which time newly 
allocated CommitLogSegments will not allow CDC
+data until a consumer parses and removes data from the destination archival 
directory.
+
+Configuration
+^
+
+Enabling or disable CDC on a table
+~~
+
+CDC is enable or disable through the `cdc` table property, for instance::
+
+CREATE TABLE foo (a int, b text, PRIMARY KEY(a)) WITH cdc=true;
+
+ALTER TABLE foo WITH cdc=true;
+
+ALTER TABLE foo WITH cdc=false;
+
+cassandra.yaml parameters
+~
+
+The following `cassandra.yaml` are available for CDC:
+
+``cdc_enabled`` (default: false)
+   Enable or disable CDC operations node-wide.
+``cdc_raw_directory`` (default: ``$CASSANDRA_HOME/data/cdc_raw``)
+   Destination for CommitLogSegments to be moved after all corresponding 
memtables are flushed.
+``cdc_free_space_in_mb``: (default: min of 4096 and 1/8th volume space)
+   Calculated as sum of all active CommitLogSegments that permit CDC + all 
flushed CDC segments in
+   ``cdc_raw_directory``.
+``cdc_free_space_check_interval_ms`` (default: 250)
+   When at capacity, we limit the frequency with which we re-calculate the 
space taken up by ``cdc_raw_directory`` to
+   prevent burning CPU cycles unnecessarily. Default is to check 4 times per 
second.
+
+.. _reading-commitlogsegments:
+
+Reading CommitLogSegments
+^
+This implementation included a refactor of CommitLogReplayer into 
`CommitLogReader.java
+`__.
+Usage is `fairly straightforward
+`__
+with a `variety of signatu

[05/34] cassandra git commit: Fill in Replication, Tuneable Consistency sections

[slebresne]

Fill in Replication, Tuneable Consistency sections


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b1edbd12
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b1edbd12
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b1edbd12

Branch: refs/heads/trunk
Commit: b1edbd12146b483516eaf3a90745ac664f46d609
Parents: 62e3d7d
Author: Tyler Hobbs 
Authored: Wed Jun 15 17:23:11 2016 -0500
Committer: Sylvain Lebresne 
Committed: Thu Jun 16 12:23:52 2016 +0200

--
 doc/source/architecture.rst | 96 +++-
 1 file changed, 94 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b1edbd12/doc/source/architecture.rst
--
diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst
index 37a0027..3f8a8ca 100644
--- a/doc/source/architecture.rst
+++ b/doc/source/architecture.rst
@@ -43,12 +43,104 @@ Token Ring/Ranges
 Replication
 ^^^
 
-.. todo:: todo
+The replication strategy of a keyspace determines which nodes are replicas for 
a given token range. The two main
+replication strategies are :ref:`simple-strategy` and 
:ref:`network-topology-strategy`.
+
+.. _simple-strategy:
+
+SimpleStrategy
+~~
+
+SimpleStrategy allows a single integer ``replication_factor`` to be defined. 
This determines the number of nodes that
+should contain a copy of each row.  For example, if ``replication_factor`` is 
3, then three different nodes should store
+a copy of each row.
+
+SimpleStrategy treats all nodes identically, ignoring any configured 
datacenters or racks.  To determine the replicas
+for a token range, Cassandra iterates through the tokens in the ring, starting 
with the token range of interest.  For
+each token, it checks whether the owning node has been added to the set of 
replicas, and if it has not, it is added to
+the set.  This process continues until ``replication_factor`` distinct nodes 
have been added to the set of replicas.
+
+.. _network-topology-strategy:
+
+NetworkTopologyStrategy
+~~~
+
+NetworkTopologyStrategy allows a replication factor to be specified for each 
datacenter in the cluster.  Even if your
+cluster only uses a single datacenter, NetworkTopologyStrategy should be 
prefered over SimpleStrategy to make it easier
+to add new physical or virtual datacenters to the cluster later.
+
+In addition to allowing the replication factor to be specified per-DC, 
NetworkTopologyStrategy also attempts to choose
+replicas within a datacenter from different racks.  If the number of racks is 
greater than or equal to the replication
+factor for the DC, each replica will be chosen from a different rack.  
Otherwise, each rack will hold at least one
+replica, but some racks may hold more than one. Note that this rack-aware 
behavior has some potentially `surprising
+implications `_.  For 
example, if there are not an even number of
+nodes in each rack, the data load on the smallest rack may be much higher.  
Similarly, if a single node is bootstrapped
+into a new rack, it will be considered a replica for the entire ring.  For 
this reason, many operators choose to
+configure all nodes on a single "rack".
 
 Tunable Consistency
 ^^^
 
-.. todo:: todo
+Cassandra supports a per-operation tradeoff between consistency and 
availability through *Consistency Levels*.
+Essentially, an operation's consistency level specifies how many of the 
replicas need to respond to the coordinator in
+order to consider the operation a success.
+
+The following consistency levels are available:
+
+``ONE``
+  Only a single replica must respond.
+
+``TWO``
+  Two replicas must respond.
+
+``THREE``
+  Three replicas must respond.
+
+``QUORUM``
+  A majority (n/2 + 1) of the replicas must respond.
+
+``ALL``
+  All of the replicas must respond.
+
+``LOCAL_QUORUM``
+  A majority of the replicas in the local datacenter (whichever datacenter the 
coordinator is in) must respond.
+
+``EACH_QUORUM``
+  A majority of the replicas in each datacenter must respond.
+
+``LOCAL_ONE``
+  Only a single replica must respond.  In a multi-datacenter cluster, this 
also gaurantees that read requests are not
+  sent to replicas in a remote datacenter.
+
+``ANY``
+  A single replica may respond, or the coordinator may store a hint. If a hint 
is stored, the coordinator will later
+  attempt to replay the hint and deliver the mutation to the replicas.  This 
consistency level is only accepted for
+  write operations.
+
+Write operations are always sent to all replicas, regardless of consistency 
level. The consistency level simply
+controls how many responses the coordinator waits f

[18/34] cassandra git commit: Reorganize document

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54f7335c/doc/source/operating/metrics.rst
--
diff --git a/doc/source/operating/metrics.rst b/doc/source/operating/metrics.rst
new file mode 100644
index 000..5884cad
--- /dev/null
+++ b/doc/source/operating/metrics.rst
@@ -0,0 +1,619 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+
+.. highlight:: none
+
+Monitoring
+--
+
+Metrics in Cassandra are managed using the `Dropwizard Metrics 
`__ library. These metrics
+can be queried via JMX or pushed to external monitoring systems using a number 
of `built in
+`__ and 
`third party
+`__ reporter plugins.
+
+Metrics are collected for a single node. It's up to the operator to use an 
external monitoring system to aggregate them.
+
+Metric Types
+
+All metrics reported by cassandra fit into one of the following types.
+
+``Gauge``
+An instantaneous measurement of a value.
+
+``Counter``
+A gauge for an ``AtomicLong`` instance. Typically this is consumed by 
monitoring the change since the last call to
+see if there is a large increase compared to the norm.
+
+``Histogram``
+Measures the statistical distribution of values in a stream of data.
+
+In addition to minimum, maximum, mean, etc., it also measures median, 
75th, 90th, 95th, 98th, 99th, and 99.9th
+percentiles.
+
+``Timer``
+Measures both the rate that a particular piece of code is called and the 
histogram of its duration.
+
+``Latency``
+Special type that tracks latency (in microseconds) with a ``Timer`` plus a 
``Counter`` that tracks the total latency
+accrued since starting. The former is useful if you track the change in 
total latency since the last check. Each
+metric name of this type will have 'Latency' and 'TotalLatency' appended 
to it.
+
+``Meter``
+A meter metric which measures mean throughput and one-, five-, and 
fifteen-minute exponentially-weighted moving
+average throughputs.
+
+Table Metrics
+^
+
+Each table in Cassandra has metrics responsible for tracking its state and 
performance.
+
+The metric names are all appended with the specific ``Keyspace`` and ``Table`` 
name.
+
+Reported name format:
+
+**Metric Name**
+
``org.apache.cassandra.metrics.Table.{{MetricName}}.{{Keyspace}}.{{Table}}``
+
+**JMX MBean**
+``org.apache.cassandra.metrics:type=Table keyspace={{Keyspace} 
scope={{Table}} name={{MetricName}}``
+
+.. NOTE::
+There is a special table called '``all``' without a keyspace. This 
represents the aggregation of metrics across
+**all** tables and keyspaces on the node.
+
+
+=== == ===
+NameType   Description
+=== == ===
+MemtableOnHeapSize  GaugeTotal amount of data 
stored in the memtable that resides **on**-heap, including column related 
overhead and partitions overwritten.
+MemtableOffHeapSize GaugeTotal amount of data 
stored in the memtable that resides **off**-heap, including column related 
overhead and partitions overwritten.
+MemtableLiveDataSizeGaugeTotal amount of live 
data stored in the memtable, excluding any data structure overhead.
+AllMemtablesOnHeapSize  GaugeTotal amount of data 
stored in the memtables (2i and pending flush memtables included) that resides 
**on**-heap.
+AllMemtablesOffHeapSize GaugeTotal amount of data 
stored in the memtables (2i and pending flush memtables included) that resides 
**off**-heap.
+AllMemtablesLiveDataSizeGaugeTotal amount of live 
data stored in the memtables (2i and pending flush memtables included) that 
resides off-heap, excluding any data structure overhead.
+MemtableColumnsCountGaugeTotal number of columns 
present in the memtable.
+MemtableSwitchCount CounterNum

[09/34] cassandra git commit: Add Metrics/Monitoring docs

[slebresne]

Add Metrics/Monitoring docs


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0ae84853
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0ae84853
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0ae84853

Branch: refs/heads/trunk
Commit: 0ae84853f84d915f5206f9d2abbeab1f66923ccb
Parents: 51b939c
Author: T Jake Luciani 
Authored: Mon Jun 20 14:09:57 2016 -0400
Committer: Sylvain Lebresne 
Committed: Tue Jun 21 14:12:59 2016 +0200

--
 doc/source/_static/extra.css |  12 +
 doc/source/operations.rst| 595 +-
 2 files changed, 605 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ae84853/doc/source/_static/extra.css
--
diff --git a/doc/source/_static/extra.css b/doc/source/_static/extra.css
index ec6aa3f..ff9f1d1 100644
--- a/doc/source/_static/extra.css
+++ b/doc/source/_static/extra.css
@@ -17,3 +17,15 @@ a.reference.internal code.literal {
 a.reference.internal:visited code.literal {
 color: #9B59B6;
 }
+
+
+/* override table width restrictions */
+.wy-table-responsive table td, .wy-table-responsive table th {
+white-space: normal;
+}
+
+.wy-table-responsive {
+margin-bottom: 24px;
+max-width: 100%;
+overflow: visible;
+}

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0ae84853/doc/source/operations.rst
--
diff --git a/doc/source/operations.rst b/doc/source/operations.rst
index d7fcafb..31ecc35 100644
--- a/doc/source/operations.rst
+++ b/doc/source/operations.rst
@@ -833,13 +833,604 @@ Backups
 Monitoring
 --
 
+Metrics in Cassandra are managed using the `Dropwizard Metrics 
`__ library. These metrics
+can be queried via JMX or pushed to external monitoring systems using a number 
of `built in
+`__ and 
`third party
+`__ reporter plugins.
+
+Metrics are collected for a single node. It's up to the operator to use an 
external monitoring system to aggregate them.
+
+Metric Types
+
+All metrics reported by cassandra fit into one of the following types.
+
+``Gauge``
+An instantaneous measurement of a value.
+
+``Counter``
+A gauge for an ``AtomicLong`` instance. Typically this is consumed by 
monitoring the change since the last call to
+see if there is a large increase compared to the norm.
+
+``Histogram``
+Measures the statistical distribution of values in a stream of data.
+
+In addition to minimum, maximum, mean, etc., it also measures median, 
75th, 90th, 95th, 98th, 99th, and 99.9th
+percentiles.
+
+``Timer``
+Measures both the rate that a particular piece of code is called and the 
histogram of its duration.
+
+``Latency``
+Special type that tracks latency (in microseconds) with a ``Timer`` plus a 
``Counter`` that tracks the total latency
+accrued since starting. The former is useful if you track the change in 
total latency since the last check. Each
+metric name of this type will have 'Latency' and 'TotalLatency' appended 
to it.
+
+``Meter``
+A meter metric which measures mean throughput and one-, five-, and 
fifteen-minute exponentially-weighted moving
+average throughputs.
+
+Table Metrics
+^
+
+Each table in Cassandra has metrics responsible for tracking its state and 
performance.
+
+The metric names are all appended with the specific ``Keyspace`` and ``Table`` 
name.
+
+Reported name format:
+
+**Metric Name**
+
``org.apache.cassandra.metrics.Table.{{MetricName}}.{{Keyspace}}.{{Table}}``
+
+**JMX MBean**
+``org.apache.cassandra.metrics:type=Table keyspace={{Keyspace} 
scope={{Table}} name={{MetricName}}``
+
+.. NOTE::
+There is a special table called '``all``' without a keyspace. This 
represents the aggregation of metrics across
+**all** tables and keyspaces on the node.
+
+
+=== == ===
+NameType   Description
+=== == ===
+MemtableOnHeapSize  GaugeTotal amount of data 
stored in the memtable that resides **on**-heap, including column related 
overhead and partitions overwritten.
+MemtableOffHeapSize GaugeTotal amount of data 
stored in the memtable that resides **off**-heap, including column related 
overhead and partitions overwritten.
+MemtableLiveDataSizeGaugeTotal amount of live 
data stored in the memtable, excluding any data structure overhead.
+All

[13/34] cassandra git commit: Fix CQL doc (incomplete)

[slebresne]

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d65542b/doc/source/cql.rst
--
diff --git a/doc/source/cql.rst b/doc/source/cql.rst
index 8d36258..8d185a2 100644
--- a/doc/source/cql.rst
+++ b/doc/source/cql.rst
@@ -16,15 +16,11 @@
 
 .. highlight:: sql
 
+.. _UUID: https://en.wikipedia.org/wiki/Universally_unique_identifier
+
 The Cassandra Query Language (CQL)
 ==
 
-CQL Syntax
---
-
-Preamble
-
-
 This document describes the Cassandra Query Language (CQL) [#]_. Note that 
this document describes the last version of
 the languages. However, the `changes <#changes>`_ section provides the diff 
between the different versions of CQL.
 
@@ -33,6 +29,13 @@ that reason, when used in this document, these terms 
(tables, rows and columns)
 in SQL. But please note that as such, they do **not** refer to the concept of 
rows and columns found in the deprecated
 thrift API (and earlier version 1 and 2 of CQL).
 
+.. _definitions:
+
+Definitions
+---
+
+.. _conventions:
+
 Conventions
 ^^^
 
@@ -41,12 +44,18 @@ To aid in specifying the CQL syntax, we will use the 
following conventions in th
 - Language rules will be given in an informal `BNF variant
   `_ notation. 
In particular, we'll use square brakets
   (``[ item ]``) for optional items, ``*`` and ``+`` for repeated items (where 
``+`` imply at least one).
-- The grammar is provided for documentation purposes and leave some minor 
details out (only conveniences that can be
-  ignored). For instance, the comma on the last column definition in a 
``CREATE TABLE`` statement is optional but
-  supported if present even though the grammar in this document suggests 
otherwise. Also, not everything accepted by the
-  grammar will be valid CQL.
+- The grammar will also use the following convention for convenience: 
non-terminal term will be lowercase (and link to
+  their definition) while terminal keywords will be provided "all caps". Note 
however that keywords are
+  :ref:`identifiers` and are thus case insensitive in practice. We will also 
define some early construction using
+  regexp, which we'll indicate with ``re()``.
+- The grammar is provided for documentation purposes and leave some minor 
details out.  For instance, the comma on the
+  last column definition in a ``CREATE TABLE`` statement is optional but 
supported if present even though the grammar in
+  this document suggests otherwise. Also, not everything accepted by the 
grammar is necessarily valid CQL.
 - References to keywords or pieces of CQL code in running text will be shown 
in a ``fixed-width font``.
 
+
+.. _identifiers:
+
 Identifiers and keywords
 
 
@@ -54,7 +63,7 @@ The CQL language uses *identifiers* (or *names*) to identify 
tables, columns and
 matching the regular expression ``[a-zA-Z][a-zA-Z0-9_]*``.
 
 A number of such identifiers, like ``SELECT`` or ``WITH``, are *keywords*. 
They have a fixed meaning for the language
-and most are reserved. The list of those keywords can be found in `Appendix A 
<#appendixA>`__.
+and most are reserved. The list of those keywords can be found in 
:ref:`appendix-A`.
 
 Identifiers and (unquoted) keywords are case insensitive. Thus ``SELECT`` is 
the same than ``select`` or ``sElEcT``, and
 ``myId`` is the same than ``myid`` or ``MYID``. A convention often used (in 
particular by the samples of this
@@ -69,1203 +78,1611 @@ sensitive (``"My Quoted Id"`` is *different* from ``"my 
quoted id"``). A fully l
 ``"myid"`` is equivalent to ``myid`` and to ``myId`` but different from 
``"myId"``).  Inside a quoted identifier, the
 double-quote character can be repeated to escape it, so ``"foo "" bar"`` is a 
valid identifier.
 
-**Warning**: *quoted identifiers* allows to declare columns with arbitrary 
names, and those can sometime clash with
-specific names used by the server. For instance, when using conditional 
update, the server will respond with a
-result-set containing a special result named ``"[applied]"``. If you’ve 
declared a column with such a name, this could
-potentially confuse some tools and should be avoided. In general, unquoted 
identifiers should be preferred but if you
-use quoted identifiers, it is strongly advised to avoid any name enclosed by 
squared brackets (like ``"[applied]"``) and
-any name that looks like a function call (like ``"f(x)"``).
+.. note:: *quoted identifiers* allows to declare columns with arbitrary names, 
and those can sometime clash with
+   specific names used by the server. For instance, when using conditional 
update, the server will respond with a
+   result-set containing a special result named ``"[applied]"``. If you’ve 
declared a column with such a name, this
+   could potentially confuse some tools and should be avoided. In general, 
unquoted identifiers should be preferred but

[06/34] cassandra git commit: Add docs for cqlsh

[slebresne]

Add docs for cqlsh


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/62e3d7dd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/62e3d7dd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/62e3d7dd

Branch: refs/heads/trunk
Commit: 62e3d7dd2cfebc18029900983272b3ef076d81b3
Parents: 4d44402
Author: Tyler Hobbs 
Authored: Wed Jun 15 14:48:27 2016 -0500
Committer: Sylvain Lebresne 
Committed: Thu Jun 16 12:23:52 2016 +0200

--
 doc/source/cqlsh.rst   | 447 
 doc/source/getting_started.rst |  19 +-
 doc/source/index.rst   |   1 +
 3 files changed, 466 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/62e3d7dd/doc/source/cqlsh.rst
--
diff --git a/doc/source/cqlsh.rst b/doc/source/cqlsh.rst
new file mode 100644
index 000..4a1b716
--- /dev/null
+++ b/doc/source/cqlsh.rst
@@ -0,0 +1,447 @@
+.. highlight:: none
+
+.. _cqlsh:
+
+cqlsh
+=
+
+cqlsh is a command line shell for interacting with Cassandra through CQL (the 
Cassandra Query Language).  It is shipped
+with every Cassandra package, and can be found in the bin/ directory alongside 
the cassandra executable.  cqlsh utilizes
+the Python native protocol driver, and connects to the single node specified 
on the command line.
+
+
+Compatibility
+-
+
+cqlsh is compatible with Python 2.7.
+
+In general, a given version of cqlsh is only guaranteed to work with the 
version of Cassandra that it was released with.
+In some cases, cqlsh make work with older or newer versions of Cassandra, but 
this is not officially supported.
+
+
+Optional Dependencies
+-
+
+cqlsh ships with all essential dependencies.  However, there are some optional 
dependencies that can be installed to
+improve the capabilities of cqlsh.
+
+pytz
+
+By default, cqlsh displays all timestamps with a UTC timezone.  To support 
display of timestamps with another timezone,
+the `pytz `__ library must be installed.  See 
the ``timezone`` option in cqlshrc_ for
+specifying a timezone to use.
+
+cython
+^^
+The performance of cqlsh's ``COPY`` operations can be improved by installing 
`cython `__.  This will
+compile the python modules that are central to the performance of ``COPY``.
+
+cqlshrc
+---
+
+The ``cqlshrc`` file holds configuration options for cqlsh.  By default this 
is in the user's home directory at
+``~/.cassandra/cqlsh``, but a custom location can be specified with the 
``--cqlshrc`` option.
+
+Example config values and documentation can be found in the 
``conf/cqlshrc.sample`` file of a tarball installation.  You
+can also view the latest version of `cqlshrc online 
`__.
+
+
+Command Line Options
+
+Usage:
+
+``cqlsh [options] [host [port]]``
+
+Options:
+
+``-C`` ``--color``
+  Force color output
+
+``--no-color``
+  Disable color output
+
+``--browser``
+  Specify the browser to use for displaying cqlsh help.  This can be one of 
the `supported browser names
+  `__ (e.g. ``firefox``) or 
a browser path followed by ``%s`` (e.g.
+  ``/usr/bin/google-chrome-stable %s``).
+
+``--ssl``
+  Use SSL when connecting to Cassandra
+
+``-u`` ``--user``
+  Username to authenticate against Cassandra with
+
+``-p`` ``--password``
+  Password to authenticate against Cassandra with, should
+  be used in conjunction with ``--user``
+
+``-k`` ``--keyspace``
+  Keyspace to authenticate to, should be used in conjunction
+  with ``--user``
+
+``-f`` ``--file``
+  Execute commands from the given file, then exit
+
+``--debug``
+  Print additional debugging information
+
+``--encoding``
+  Specify a non-default encoding for output (defaults to UTF-8)
+
+``--cqlshrc``
+  Specify a non-default location for the ``cqlshrc`` file
+
+``-e`` ``--execute``
+  Execute the given statement, then exit
+
+``--connect-timeout``
+  Specify the connection timeout in seconds (defaults to 2s)
+
+``--request-timeout``
+  Specify the request timeout in seconds (defaults to 10s)
+
+``-t`` ``--tty``
+  Force tty mode (command prompt)
+
+
+Special Commands
+
+
+In addition to supporting regular CQL statements, cqlsh also supports a number 
of special commands that are not part of
+CQL.  These are detailed below.
+
+``CONSISTENCY``
+^^^
+
+`Usage`: ``CONSISTENCY ``
+
+Sets the consistency level for operations to follow.  Valid arguments include:
+
+- ``ANY``
+- ``ONE``
+- ``TWO``
+- ``THREE``
+- ``QUORUM``
+- ``ALL``
+- ``LOCAL_QUORUM``
+- ``LOCAL_ONE``
+- ``SERIAL``
+- ``LOCAL_SERIAL``
+
+

[03/34] cassandra git commit: Add initial in-tree documentation (very incomplete so far)

[slebresne]

Add initial in-tree documentation (very incomplete so far)

patch by Stefania, jjirsa, rspitzer & slebresne; reviewed by slebresne for 
CASSANDRA-8700


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cad277be
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cad277be
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cad277be

Branch: refs/heads/trunk
Commit: cad277be41ad4d5488cb96c5ef3f2fcb1cc8d3d9
Parents: 719e7d6
Author: Sylvain Lebresne 
Authored: Tue Jun 14 14:45:16 2016 +0200
Committer: Sylvain Lebresne 
Committed: Tue Jun 14 18:16:42 2016 +0200

--
 build.xml  |   13 +
 doc/Makefile   |  225 ++
 doc/README.md  |   31 +
 doc/make.bat   |  281 +++
 doc/source/_static/extra.css   |   19 +
 doc/source/architecture.rst|   74 +
 doc/source/conf.py |  430 
 doc/source/contactus.rst   |   49 +
 doc/source/cql.rst | 3916 +++
 doc/source/faq.rst |   20 +
 doc/source/getting_started.rst |  252 +++
 doc/source/index.rst   |   35 +
 doc/source/operations.rst  |  369 
 doc/source/troubleshooting.rst |   20 +
 14 files changed, 5734 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cad277be/build.xml
--
diff --git a/build.xml b/build.xml
index aa55809..51f7252 100644
--- a/build.xml
+++ b/build.xml
@@ -67,6 +67,8 @@
 
 
 
+
+

 
 
@@ -249,6 +251,17 @@
 
 
 
+
+
+
+
+
+
+
+
+
+
+
 

[jira] [Updated] (CASSANDRA-11968) More metrics on native protocol requests & responses

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11968:
---
Status: Open  (was: Patch Available)

> More metrics on native protocol requests & responses
> 
>
> Key: CASSANDRA-11968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11968
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.x
>
>
> Proposal to add more metrics to the native protocol:
> - number of requests per request-type
> - number of responses by response-type
> - size of request messages in bytes
> - size of response messages in bytes
> - number of in-flight requests (from request arrival to response)
> (Will provide a patch soon)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11972) Use byte[] instead of object tree in Frame.Header

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-11972.
---
Resolution: Won't Fix

> Use byte[] instead of object tree in Frame.Header
> -
>
> Key: CASSANDRA-11972
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11972
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
>
> Replacing the object tree/references in {{Frame.Header}} with {{byte[9]}} 
> saves a couple of object allocations. Also, not allocating the 9 bytes for 
> the header off-heap is less expensive.
> (will provide a patch soon)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-11972) Use byte[] instead of object tree in Frame.Header

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko reopened CASSANDRA-11972:
---

> Use byte[] instead of object tree in Frame.Header
> -
>
> Key: CASSANDRA-11972
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11972
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
>
> Replacing the object tree/references in {{Frame.Header}} with {{byte[9]}} 
> saves a couple of object allocations. Also, not allocating the 9 bytes for 
> the header off-heap is less expensive.
> (will provide a patch soon)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11580) remove DatabaseDescriptor dependency from SegmentedFile

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11580:
---
Status: Ready to Commit  (was: Patch Available)

> remove DatabaseDescriptor dependency from SegmentedFile
> ---
>
> Key: CASSANDRA-11580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11580
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
> Fix For: 3.x
>
>
> Several configurable parameters are pulled from {{DatabaseDescriptor}} from 
> {{SegmentedFile}} and its subclasses.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11972) Use byte[] instead of object tree in Frame.Header

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11972:
--
Fix Version/s: (was: 3.x)

> Use byte[] instead of object tree in Frame.Header
> -
>
> Key: CASSANDRA-11972
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11972
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
>
> Replacing the object tree/references in {{Frame.Header}} with {{byte[9]}} 
> saves a couple of object allocations. Also, not allocating the 9 bytes for 
> the header off-heap is less expensive.
> (will provide a patch soon)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11907) 2i behaviour is different in different versions

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11907:
---
Status: Open  (was: Patch Available)

> 2i behaviour is different in different versions
> ---
>
> Key: CASSANDRA-11907
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11907
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Tommy Stendahl
>Assignee: Alex Petrov
>
>  I think I have found more cases where 2i behave different in different 
> Cassandra versions, CASSANDRA-11510 solved one such case but I think there 
> are a few more.
> I get one behaviour with 2.1.14 and Trunk and I think this is the correct 
> one. With 2.2.7 and 3.0.6 the behaviour is different.
> To test this I used ccm to setup one node clusters with the different 
> versions, I prepared each cluster with these commands:
> {code:sql}
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 
> 'datacenter1': '1' };
> CREATE TABLE test.table1 (name text,class int,inter text,foo text,power 
> int,PRIMARY KEY (name, class, inter, foo)) WITH CLUSTERING ORDER BY (class 
> DESC, inter ASC);
> CREATE INDEX table1_power ON test.table1 (power) ;
> CREATE TABLE test.table2 (name text,class int,inter text,foo text,power 
> int,PRIMARY KEY (name, class, inter, foo)) WITH CLUSTERING ORDER BY (class 
> DESC, inter ASC);
> CREATE INDEX table2_inter ON test.table2 (inter) ;
> {code}
> I executed two select quieries on each cluster:
> {code:sql}
> SELECT * FROM test.table1 where name='R1' AND class>0 AND class<4 AND 
> inter='int1' AND power=18 ALLOW FILTERING;
> SELECT * FROM test.table2 where name='R1' AND class>0 AND class<4 AND 
> inter='int1' AND foo='aa' ALLOW FILTERING;
> {code}
> On 2.1.14 and Trunk they where successful. But on 2.2.7 and 3.0.6 they 
> failed, the first one with {{InvalidRequest: code=2200 [Invalid query] 
> message="Clustering column "inter" cannot be restricted (preceding column 
> "class" is restricted by a non-EQ relation)"}} and the second one with 
> {{InvalidRequest: code=2200 [Invalid query] message="Clustering column "foo" 
> cannot be restricted (preceding column "inter" is restricted by a non-EQ 
> relation)"}}.
> I could get the queries to execute successfully on 2.2.7 and 3.0.6 by 
> creating two more 2i:
> {code:sql}
> CREATE INDEX table1_inter ON test.table1 (inter) ;
> CREATE INDEX table2_foo ON test.table2 (foo) ;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11972) Use byte[] instead of object tree in Frame.Header

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351495#comment-15351495
 ] 

Jonathan Ellis commented on CASSANDRA-11972:


Resolving as wontfix given the above comments, and given that I'm trying to 
reduce the Patch Available backlog.  Feel free to reopen if profiling or 
benchmarking shows that this is more important than expected.

> Use byte[] instead of object tree in Frame.Header
> -
>
> Key: CASSANDRA-11972
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11972
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.x
>
>
> Replacing the object tree/references in {{Frame.Header}} with {{byte[9]}} 
> saves a couple of object allocations. Also, not allocating the 9 bytes for 
> the header off-heap is less expensive.
> (will provide a patch soon)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11972) Use byte[] instead of object tree in Frame.Header

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11972:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Use byte[] instead of object tree in Frame.Header
> -
>
> Key: CASSANDRA-11972
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11972
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.x
>
>
> Replacing the object tree/references in {{Frame.Header}} with {{byte[9]}} 
> saves a couple of object allocations. Also, not allocating the 9 bytes for 
> the header off-heap is less expensive.
> (will provide a patch soon)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11393) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351449#comment-15351449
 ] 

Russ Hatch commented on CASSANDRA-11393:


OK, I'll put together a dtest branch to test:

2.1.latest -> your 3.0 branch
2.2.latest -> your 3.0 branch
2.1.latest -> your 3.7 branch
2.2.latest -> your 3.7 branch

Any other useful cases?

> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11393
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Streaming and Messaging
>Reporter: Philip Thompson
>Assignee: Benjamin Lerer
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: 11393-3.0.txt
>
>
> We are seeing a failure in the upgrade tests that go from 2.1 to 3.0
> {code}
> node2: ERROR [SharedPool-Worker-2] 2016-03-10 20:05:17,865 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xeb79b477, 
> /127.0.0.1:39613 => /127.0.0.2:9042]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1208)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1155)
>  ~[main/:na]
>   at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor$AlwaysSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:330)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.SinglePartitionReadCommand.execute(SinglePartitionReadCommand.java:302)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:67)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.SinglePartitionPager.fetchPage(SinglePartitionPager.java:34)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:297)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:333)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:209)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:472)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:449)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [n

[jira] [Commented] (CASSANDRA-11393) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351420#comment-15351420
 ] 

Russ Hatch commented on CASSANDRA-11393:


I'm not familiar with that setting, is it something we should have set by 
default on the regular upgrade jobs?

> dtest failure in 
> upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_2_1_UpTo_3_0_HEAD.rolling_upgrade_test
> --
>
> Key: CASSANDRA-11393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11393
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Streaming and Messaging
>Reporter: Philip Thompson
>Assignee: Benjamin Lerer
>  Labels: dtest
> Fix For: 3.0.x, 3.x
>
> Attachments: 11393-3.0.txt
>
>
> We are seeing a failure in the upgrade tests that go from 2.1 to 3.0
> {code}
> node2: ERROR [SharedPool-Worker-2] 2016-03-10 20:05:17,865 Message.java:611 - 
> Unexpected exception during request; channel = [id: 0xeb79b477, 
> /127.0.0.1:39613 => /127.0.0.2:9042]
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1208)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1155)
>  ~[main/:na]
>   at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.AbstractReadExecutor$AlwaysSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:330)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) 
> ~[main/:na]
>   at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.SinglePartitionReadCommand.execute(SinglePartitionReadCommand.java:302)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:67)
>  ~[main/:na]
>   at 
> org.apache.cassandra.service.pager.SinglePartitionPager.fetchPage(SinglePartitionPager.java:34)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:297)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:333)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:209)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:472)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:449)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractC

[jira] [Commented] (CASSANDRA-11854) Remove finished streaming connections from MessagingService

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351418#comment-15351418
 ] 

Paulo Motta commented on CASSANDRA-11854:
-

bq. In SocketThread#close() we iterate over the connections and close each one, 
this will not work now since the connections will be empty, right? 

Actually this would only be empty if there are no ongoing stream sessions, 
otherwise active stream sessions would only be cleared after the 
{{StreamResultFuture}} was completed.

bq. Should we perhaps close() the socket when we remove it from group? They 
were only closed on server shutdown before so maybe we don't need to do that?

It's not safe to close the socket when a stream session is finished (failed or 
completed), because the outgoing stream handler may still need to send queued 
{{SessionComplete}} or {{SessionFailed}} messages after the 
{{StreamResultFuture}} returns. For this reason, there may be edge cases with 
the previous approach when there are still active streaming connections not 
closed by {{MessagingService.shutdown()}}.

In order to guarantee streaming sockets are only removed from 
{{SocketThread.connections}} after the {{(Incoming|Outgoing)MessageHandler}} 
are finished, the new approach attaches the {{IncomingStreamingConnection}} to 
the {{ConnectionHandler}}, so on {{ConnectionHandler.signalCloseDone()}} the 
{{IncomingStreamingConnection}} is closed, closing the socket and removing it 
from {{SocketThread.connections}}. I also added a unit test to verify that all 
sockets are cleared from {{SocketThread.connections}} after the stream session 
finishes.

New patch and tests below:

||2.1||2.2||3.0||trunk||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.1...pauloricardomg:2.1-11854-v3]|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-11854-v3]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-11854-v3]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-11854-v3]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-11854-v3-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-11854-v3-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-11854-v3-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-11854-v3-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-11854-v3-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-11854-v3-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-11854-v3-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-11854-v3-dtest/lastCompletedBuild/testReport/]|

commit info: minor conflicts all the way up to trunk :(

> Remove finished streaming connections from MessagingService
> ---
>
> Key: CASSANDRA-11854
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11854
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Paulo Motta
>Assignee: Paulo Motta
> Attachments: oom.png
>
>
> When a new {{IncomingStreamingConnection}} is created, [we register it in the 
> connections 
> map|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/MessagingService.java#L1109]
>  of {{MessagingService}}, but we [only remove it if there is an 
> exception|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/IncomingStreamingConnection.java#L83]
>  while attaching the socket to the stream session.
> On nodes with SSL and large number of vnodes, after many repair sessions 
> these old connections can accumulate and cause OOM (heap dump attached).
> The connection should be removed from the connections map after if it's 
> finished in order to be garbage collected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions

[Branimir Lambov (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351341#comment-15351341
 ] 

Branimir Lambov commented on CASSANDRA-9754:


I spent some time reading up {{BirchReader}} to figure out the nuts and bolts 
of how the storage works. I think we can squeeze a little more efficiency into 
the structure:
- As far as I could see, your current implementation places a lot of copies on 
the lower side of each span in the non-leaf nodes (for example, the lowest key 
of the partition is present in the leaf node, its parent as well as all parents 
leading all the way to the root). This should not be necessary, simply omitting 
the first key (but retaining the child pointer) from all intermediate nodes and 
adding 1 to what the binary search returns will achieve the same result.
- I find the overlow flag (and jumping back and forth to read it) less 
efficient than necessary. If we assume instead that key length equal to the max 
always entails overflow data, we would be using less space and be more 
efficient in the common case, while having a very low chance of taking a few 
bytes more in the uncommon situation of long keys.
- Root node could be in the same page with descriptor (it is usually smaller so 
high chance to fit). Perhaps overflow is best placed elsewhere?

More generally (ignoring padding on the leaves which is not necessarily always 
beneficial), the B+ structure you have built is practically a B-Tree index over 
a linear list of index entries. As we already have a linear list of 
{{IndexInfo}} structures in the current format, what are we gaining by not just 
building a B-Tree index over that? To me the latter would appear to be less 
complicated and much more generic with immediate possible applications in other 
parts of the codebase.


> Make index info heap friendly for large CQL partitions
> --
>
> Key: CASSANDRA-9754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9754
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Assignee: Michael Kjellman
>Priority: Minor
> Attachments: 9754_part1-v1.diff, 9754_part2-v1.diff
>
>
>  Looking at a heap dump of 2.0 cluster, I found that majority of the objects 
> are IndexInfo and its ByteBuffers. This is specially bad in endpoints with 
> large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K 
> IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for 
> GC. Can this be improved by not creating so many objects?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10928) SSTableExportTest.testExportColumnsWithMetadata randomly fails

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351337#comment-15351337
 ] 

Jonathan Ellis commented on CASSANDRA-10928:


I hit the same error running this test locally.  I pulled in the slightly more 
elegant fix from 2.2 and committed.

{code}
assertEquals("unexpected serialization format for topLevelDeletion",
 
JSONValue.parse("{\"markedForDeleteAt\":0,\"localDeletionTime\":0}"),
 serializedDeletionInfo);
{code}

> SSTableExportTest.testExportColumnsWithMetadata randomly fails
> --
>
> Key: CASSANDRA-10928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10928
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Michael Kjellman
>Assignee: sankalp kohli
> Fix For: 2.1.x
>
> Attachments: CASSANDRA_10928_2.1.diff
>
>
> The SSTableExportTest.testExportColumnsWithMetadata test will randomly fail 
> (bogusly). Currently, the string check used won’t work if the JSON generated 
> happened to order the elements in the array differently.
> {code}
> assertEquals(
> "unexpected serialization format for topLevelDeletion",
> "{\"markedForDeleteAt\":0,\"localDeletionTime\":0}",
> serializedDeletionInfo.toJSONString());
> {code}
> {noformat}
> [junit] Testcase: 
> testExportColumnsWithMetadata(org.apache.cassandra.tools.SSTableExportTest):  
>   FAILED
> [junit] unexpected serialization format for topLevelDeletion 
> expected:<{"[markedForDeleteAt":0,"localDeletionTime]":0}> but 
> was:<{"[localDeletionTime":0,"markedForDeleteAt]":0}>
> [junit] junit.framework.AssertionFailedError: unexpected serialization 
> format for topLevelDeletion 
> expected:<{"[markedForDeleteAt":0,"localDeletionTime]":0}> but 
> was:<{"[localDeletionTime":0,"markedForDeleteAt]":0}>
> [junit]   at 
> org.apache.cassandra.tools.SSTableExportTest.testExportColumnsWithMetadata(SSTableExportTest.java:299)
> [junit]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10928) SSTableExportTest.testExportColumnsWithMetadata randomly fails

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-10928:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> SSTableExportTest.testExportColumnsWithMetadata randomly fails
> --
>
> Key: CASSANDRA-10928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10928
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Michael Kjellman
>Assignee: sankalp kohli
> Fix For: 2.1.x
>
> Attachments: CASSANDRA_10928_2.1.diff
>
>
> The SSTableExportTest.testExportColumnsWithMetadata test will randomly fail 
> (bogusly). Currently, the string check used won’t work if the JSON generated 
> happened to order the elements in the array differently.
> {code}
> assertEquals(
> "unexpected serialization format for topLevelDeletion",
> "{\"markedForDeleteAt\":0,\"localDeletionTime\":0}",
> serializedDeletionInfo.toJSONString());
> {code}
> {noformat}
> [junit] Testcase: 
> testExportColumnsWithMetadata(org.apache.cassandra.tools.SSTableExportTest):  
>   FAILED
> [junit] unexpected serialization format for topLevelDeletion 
> expected:<{"[markedForDeleteAt":0,"localDeletionTime]":0}> but 
> was:<{"[localDeletionTime":0,"markedForDeleteAt]":0}>
> [junit] junit.framework.AssertionFailedError: unexpected serialization 
> format for topLevelDeletion 
> expected:<{"[markedForDeleteAt":0,"localDeletionTime]":0}> but 
> was:<{"[localDeletionTime":0,"markedForDeleteAt]":0}>
> [junit]   at 
> org.apache.cassandra.tools.SSTableExportTest.testExportColumnsWithMetadata(SSTableExportTest.java:299)
> [junit]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: backport fix for flaky testExportColumnsWithMetadata from 2.2

[jbellis]

backport fix for flaky testExportColumnsWithMetadata from 2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/341b3fbf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/341b3fbf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/341b3fbf

Branch: refs/heads/cassandra-2.2
Commit: 341b3fbfb444bc0af81700cdc5a30d5bafea04a1
Parents: 9358e58
Author: Jonathan Ellis 
Authored: Mon Jun 27 11:13:48 2016 -0500
Committer: Jonathan Ellis 
Committed: Mon Jun 27 11:13:48 2016 -0500

--
 test/unit/org/apache/cassandra/tools/SSTableExportTest.java | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/341b3fbf/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
--
diff --git a/test/unit/org/apache/cassandra/tools/SSTableExportTest.java 
b/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
index c918d6a..16f4bda 100644
--- a/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
+++ b/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
@@ -296,10 +296,9 @@ public class SSTableExportTest extends SchemaLoader
 JSONObject serializedDeletionInfo = (JSONObject) 
meta.get("deletionInfo");
 assertNotNull("expecing deletionInfo to be present", 
serializedDeletionInfo);
 
-assertEquals(
-"unexpected serialization format for topLevelDeletion",
-"{\"markedForDeleteAt\":0,\"localDeletionTime\":0}",
-serializedDeletionInfo.toJSONString());
+assertEquals("unexpected serialization format for topLevelDeletion",
+ 
JSONValue.parse("{\"markedForDeleteAt\":0,\"localDeletionTime\":0}"),
+ serializedDeletionInfo);
 
 // check the colums are what we put in
 JSONArray cols = (JSONArray) row.get("cells");

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2

[jbellis]

Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6fcf3353
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6fcf3353
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6fcf3353

Branch: refs/heads/cassandra-2.2
Commit: 6fcf3353b65195f0b1d0c1e20c5b8d562a8d21d5
Parents: 5d0d30e 341b3fb
Author: Jonathan Ellis 
Authored: Mon Jun 27 11:13:56 2016 -0500
Committer: Jonathan Ellis 
Committed: Mon Jun 27 11:13:56 2016 -0500

--

--

[1/3] cassandra git commit: backport fix for flaky testExportColumnsWithMetadata from 2.2

[jbellis]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 9358e589e -> 341b3fbfb
  refs/heads/cassandra-2.2 5d0d30e4d -> 6fcf3353b


backport fix for flaky testExportColumnsWithMetadata from 2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/341b3fbf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/341b3fbf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/341b3fbf

Branch: refs/heads/cassandra-2.1
Commit: 341b3fbfb444bc0af81700cdc5a30d5bafea04a1
Parents: 9358e58
Author: Jonathan Ellis 
Authored: Mon Jun 27 11:13:48 2016 -0500
Committer: Jonathan Ellis 
Committed: Mon Jun 27 11:13:48 2016 -0500

--
 test/unit/org/apache/cassandra/tools/SSTableExportTest.java | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/341b3fbf/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
--
diff --git a/test/unit/org/apache/cassandra/tools/SSTableExportTest.java 
b/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
index c918d6a..16f4bda 100644
--- a/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
+++ b/test/unit/org/apache/cassandra/tools/SSTableExportTest.java
@@ -296,10 +296,9 @@ public class SSTableExportTest extends SchemaLoader
 JSONObject serializedDeletionInfo = (JSONObject) 
meta.get("deletionInfo");
 assertNotNull("expecing deletionInfo to be present", 
serializedDeletionInfo);
 
-assertEquals(
-"unexpected serialization format for topLevelDeletion",
-"{\"markedForDeleteAt\":0,\"localDeletionTime\":0}",
-serializedDeletionInfo.toJSONString());
+assertEquals("unexpected serialization format for topLevelDeletion",
+ 
JSONValue.parse("{\"markedForDeleteAt\":0,\"localDeletionTime\":0}"),
+ serializedDeletionInfo);
 
 // check the colums are what we put in
 JSONArray cols = (JSONArray) row.get("cells");

[jira] [Created] (CASSANDRA-12099) dtest failure in snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12099:
-

 Summary: dtest failure in 
snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address
 Key: CASSANDRA-12099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12099
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log

example failure:

http://cassci.datastax.com/job/trunk_offheap_dtest/274/testReport/snitch_test/TestGossipingPropertyFileSnitch/test_prefer_local_reconnect_on_listen_address

Failed on CassCI build trunk_offheap_dtest #274

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 358, in run
self.tearDown()
  File "/home/automaton/cassandra-dtest/dtest.py", line 761, in tearDown
['\n'.join(msg) for msg in node.grep_log_for_errors()]))
  File "/home/automaton/ccm/ccmlib/node.py", line 357, in grep_log_for_errors
return self.grep_log_for_errors_from(seek_start=getattr(self, 'error_mark', 
0))
  File "/home/automaton/ccm/ccmlib/node.py", line 360, in 
grep_log_for_errors_from
with open(os.path.join(self.get_path(), 'logs', filename)) as f:
"[Errno 2] No such file or directory: 
'/mnt/tmp/dtest-Pa4WH7/test/node2/logs/system.log'
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10983) Metrics for tracking offending queries

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-10983:
---
Reviewer:   (was: Brandon Williams)

> Metrics for tracking offending queries
> --
>
> Key: CASSANDRA-10983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10983
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Sharvanath Pathak
>  Labels: github-import
> Fix For: 2.1.x
>
>
> I have seen big GC pauses leading to nodes being marked DOWN in our cluster. 
> The most common issue is someone, would add a large range scan and it would 
> be difficult to pin-point the specific query. I have added a mechanism to 
> account the memory allocation for a specific query. In order to allow 
> aggregates over a period I added a metric as well. Attached is the diff.
> I was wondering if something like this would be interesting for more general 
> audience. There are some things which need to be fixed for proper release. 
> For instance, Cleaning up existing metrics on server restart. However, just 
> wanted to check before that if something like this would be useful for others.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12098) dtest failure in secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-12098:

Issue Type: Bug  (was: Test)

> dtest failure in 
> secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query
> --
>
> Key: CASSANDRA-12098
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12098
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: DS Test Eng
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/273/testReport/secondary_indexes_test/TestSecondaryIndexes/test_only_coordinator_chooses_index_for_query
> Failed on CassCI build trunk_offheap_dtest #273
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MessagingService-Incoming-/127.0.0.3] 2016-06-26 08:11:32,185 
> CassandraDaemon.java:219 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.3,5,main]
> java.lang.RuntimeException: Unknown column b during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:642)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:349)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:368)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:305)
>  ~[main/:na]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12098) dtest failure in secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-12098:

Assignee: (was: DS Test Eng)

> dtest failure in 
> secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query
> --
>
> Key: CASSANDRA-12098
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12098
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/273/testReport/secondary_indexes_test/TestSecondaryIndexes/test_only_coordinator_chooses_index_for_query
> Failed on CassCI build trunk_offheap_dtest #273
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MessagingService-Incoming-/127.0.0.3] 2016-06-26 08:11:32,185 
> CassandraDaemon.java:219 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.3,5,main]
> java.lang.RuntimeException: Unknown column b during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:642)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:349)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:368)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:305)
>  ~[main/:na]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12098) dtest failure in secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-12098:

Fix Version/s: 3.x

> dtest failure in 
> secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query
> --
>
> Key: CASSANDRA-12098
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12098
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sean McCarthy
>  Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
> node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
>
>
> example failure:
> http://cassci.datastax.com/job/trunk_offheap_dtest/273/testReport/secondary_indexes_test/TestSecondaryIndexes/test_only_coordinator_chooses_index_for_query
> Failed on CassCI build trunk_offheap_dtest #273
> {code}
> Standard Output
> Unexpected error in node1 log, error: 
> ERROR [MessagingService-Incoming-/127.0.0.3] 2016-06-26 08:11:32,185 
> CassandraDaemon.java:219 - Exception in thread 
> Thread[MessagingService-Incoming-/127.0.0.3,5,main]
> java.lang.RuntimeException: Unknown column b during deserialization
>   at 
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:642)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:349)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:368)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:305)
>  ~[main/:na]
>   at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>  ~[main/:na]
>   at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>  ~[main/:na]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11868) unused imports and generic types

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11868:
---
Status: Open  (was: Patch Available)

> unused imports and generic types
> 
>
> Key: CASSANDRA-11868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11868
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 3.x
>
>
> I was going through Cassandra source and for busy work I started looking at 
> all the .java files eclipse flags as warning. They are broken roughly into a 
> few cases. 
> 1) unused imports 
> 2) raw types missing <> 
> 3) case statements without defaults 
> 4) @resource annotation 
> My IDE claims item 4 is not needed (it looks like we have done this to 
> signify methods that return objects that need to be closed) I can guess 4 was 
> done intentionally and short of making out own annotation I will ignore these 
> for now. 
> I would like to tackle this busy work before I get started. I have some 
> questions: 
> 1) Do this only on trunk? or multiple branches 
> 2) should I tackle 1,2,3 in separate branches/patches



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11734) Enable partition component index for SASI

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-11734:
---
Status: Open  (was: Patch Available)

> Enable partition component index for SASI
> -
>
> Key: CASSANDRA-11734
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11734
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
>  Labels: doc-impacting, sasi, secondaryIndex
> Fix For: 3.x
>
> Attachments: patch.txt
>
>
> Enable partition component index for SASI
> For the given schema:
> {code:sql}
> CREATE TABLE test.comp (
> pk1 int,
> pk2 text,
> val text,
> PRIMARY KEY ((pk1, pk2))
> );
> CREATE CUSTOM INDEX comp_val_idx ON test.comp (val) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> CREATE CUSTOM INDEX comp_pk2_idx ON test.comp (pk2) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 'PREFIX', 
> 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> CREATE CUSTOM INDEX comp_pk1_idx ON test.comp (pk1) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> The following queries are possible:
> {code:sql}
> SELECT * FROM test.comp WHERE pk1=1;
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5;
> SELECT * FROM test.comp WHERE pk1=1 AND val='xxx' ALLOW FILTERING;
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5 AND val='xxx' ALLOW FILTERING;
> SELECT * FROM test.comp WHERE pk2='some text';
> SELECT * FROM test.comp WHERE pk2 LIKE 'prefix%';
> SELECT * FROM test.comp WHERE pk2='some text' AND val='xxx' ALLOW FILTERING;
> SELECT * FROM test.comp WHERE pk2 LIKE 'prefix%' AND val='xxx' ALLOW 
> FILTERING;
> //Without using SASI
> SELECT * FROM test.comp WHERE pk1 = 1 AND pk2='some text';
> SELECT * FROM test.comp WHERE pk1 IN(1,2,3) AND pk2='some text';
> SELECT * FROM test.comp WHERE pk1 = 1 AND pk2 IN ('text1','text2');
> SELECT * FROM test.comp WHERE pk1 IN(1,2,3) AND pk2 IN ('text1','text2');
> {code}
> However, the following queries *are not possible*
> {code:sql}
> SELECT * FROM test.comp WHERE pk1=1 AND pk2 LIKE 'prefix%';
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5 AND pk2 = 'some text';
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5 AND pk2 LIKE 'prefix%';
> {code}
> All of them are throwing the following exception
> {noformat}
> ava.lang.UnsupportedOperationException: null
>   at 
> org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$LikeRestriction.appendTo(SingleColumnRestriction.java:715)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.PartitionKeySingleRestrictionSet.values(PartitionKeySingleRestrictionSet.java:86)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getPartitionKeys(StatementRestrictions.java:585)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:473)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getQuery(SelectStatement.java:265)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:230)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:79)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:208)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:239) 
> ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:224) 
> ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:283)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
>   at 
> org.apach

[jira] [Created] (CASSANDRA-12098) dtest failure in secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12098:
-

 Summary: dtest failure in 
secondary_indexes_test.TestSecondaryIndexes.test_only_coordinator_chooses_index_for_query
 Key: CASSANDRA-12098
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12098
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log

example failure:

http://cassci.datastax.com/job/trunk_offheap_dtest/273/testReport/secondary_indexes_test/TestSecondaryIndexes/test_only_coordinator_chooses_index_for_query

Failed on CassCI build trunk_offheap_dtest #273

{code}
Standard Output

Unexpected error in node1 log, error: 
ERROR [MessagingService-Incoming-/127.0.0.3] 2016-06-26 08:11:32,185 
CassandraDaemon.java:219 - Exception in thread 
Thread[MessagingService-Incoming-/127.0.0.3,5,main]
java.lang.RuntimeException: Unknown column b during deserialization
at 
org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433) 
~[main/:na]
at 
org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:407)
 ~[main/:na]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:192)
 ~[main/:na]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:668)
 ~[main/:na]
at 
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:642)
 ~[main/:na]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:349)
 ~[main/:na]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:368)
 ~[main/:na]
at 
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:305)
 ~[main/:na]
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
~[main/:na]
at 
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190)
 ~[main/:na]
at 
org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
 ~[main/:na]
at 
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
 ~[main/:na]
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-12097) dtest failure in materialized_views_test.TestMaterializedViews.view_tombstone_test

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12097:
-

 Summary: dtest failure in 
materialized_views_test.TestMaterializedViews.view_tombstone_test
 Key: CASSANDRA-12097
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12097
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log

example failure:

http://cassci.datastax.com/job/trunk_offheap_dtest/271/testReport/materialized_views_test/TestMaterializedViews/view_tombstone_test

Failed on CassCI build trunk_offheap_dtest #271

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/materialized_views_test.py", line 754, 
in view_tombstone_test
[1, 1, 'b', 3.0]
  File "/home/automaton/cassandra-dtest/assertions.py", line 51, in assert_one
assert list_res == [expected], "Expected %s from %s, but got %s" % 
([expected], query, list_res)
"Expected [[1, 1, 'b', 3.0]] from SELECT * FROM t_by_v WHERE v = 1, but got 
[[1, 1, u'b', None]]
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10594) Inconsistent permissions results return

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-10594:

Assignee: (was: Sam Tunnicliffe)

> Inconsistent permissions results return
> ---
>
> Key: CASSANDRA-10594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10594
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Adam Holmberg
>Priority: Minor
>
> The server returns inconsistent results when listing permissions, depending 
> on whether a user is configured.
> *Observed with Cassandra 3.0:*
> Only super user configured:
> {code}
> cassandra@cqlsh> list all;
>  role | resource | permissions
> --+--+-
> (0 rows)
> {code}
> VOID result type is returned (meaning no result meta is returned and cqlsh 
> must use the table meta to determine columns)
> With one user configured, no grants:
> {code}
> cassandra@cqlsh> create user holmberg with password 'tmp';
> cassandra@cqlsh> list all;
> results meta: system_auth permissions 4
>  role  | username  | resource| permission
> ---+---+-+
>  cassandra | cassandra |  |  ALTER
>  cassandra | cassandra |  |   DROP
>  cassandra | cassandra |  |  AUTHORIZE
> (3 rows)
> {code}
> Now a ROWS result message is returned with the cassandra super user grants. 
> Dropping the regular user causes the VOID message to be returned again.
> *Slightly different behavior on 2.2 branch:* VOID message with no result meta 
> is returned, even if regular user is configured, until permissions are added 
> to that user.
> *Expected:*
> It would be nice if the query always resulted in a ROWS result, even if there 
> are no explicit permissions defined. This would provide the correct result 
> metadata even if there are no rows.
> Additionally, it is strange that the 'cassandra' super user only appears in 
> the results when another user is configured. I would expect it to always 
> appear, or never.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8094) Heavy writes in RangeSlice read requests

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-8094:
---
Assignee: (was: Sam Tunnicliffe)

> Heavy writes in RangeSlice read  requests 
> --
>
> Key: CASSANDRA-8094
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8094
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Minh Do
>  Labels: lhf
> Fix For: 3.x
>
>
> RangeSlice requests always do a scheduled read repair when coordinators try 
> to resolve replicas' responses no matter read_repair_chance is set or not.
> Because of this, in low writes and high reads clusters, there are very high 
> write requests going on between nodes.  
> We should have an option to turn this off and this can be different than the 
> read_repair_chance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8941) Test Coverage for CASSANDRA-8786

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-8941:
---
Assignee: (was: Sam Tunnicliffe)

> Test Coverage for CASSANDRA-8786
> 
>
> Key: CASSANDRA-8941
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8941
> Project: Cassandra
>  Issue Type: Test
>  Components: Testing
>Reporter: Tyler Hobbs
>Priority: Minor
> Fix For: 2.1.x
>
>
> We don't currently have a test to reproduce the issue from CASSANDRA-8786.  
> It would be good to track down exactly what circustances cause this and add 
> some test coverage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10214) Enable index selection to be overridden on a per query basis

[Sam Tunnicliffe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-10214:

Assignee: (was: Sam Tunnicliffe)

> Enable index selection to be overridden on a per query basis
> 
>
> Key: CASSANDRA-10214
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10214
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Sam Tunnicliffe
> Fix For: 3.x
>
>
> (Broken out of CASSANDRA-10124)
> We could add a {{USING INDEX }} clause to {{SELECT}} syntax to 
> force the choice of index and bypass the usual index selection mechanism.
> {code}
> CREATE TABLE ks.t1(k int, v1 int, v2 int, PRIMARY KEY (k));
> CREATE INDEX v1_idx ON ks.t1(v1);
> CREATE INDEX v2_idx ON ks.t1(v2);
> CREATE CUSTOM INDEX v1_v2_idx ON ks.t1(v1, v2) USING 
> 'com.foo.bar.CustomMultiColumnIndex';
> # Override internal index selection mechanism
> SELECT * FROM ks.t1 WHERE v1=0 AND v2=0 USING INDEX v1_idx;
> SELECT * FROM ks.t1 WHERE v1=0 AND v2=0 USING INDEX v2_idx;
> SELECT * FROM ks.t1 WHERE v1=0 AND v2=0 USING INDEX v1_v2_idx;
> {code}
> This is in some ways similar to [index 
> hinting|http://docs.oracle.com/cd/B19306_01/server.102/b14211/hintsref.htm#CHDJDIAH]
>  in Oracle. 
> edit: fixed typo's (missing INDEX in the USING clauses)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12041) Add CDC to describe table

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12041:
--
Fix Version/s: (was: 3.8)
   3.x

> Add CDC to describe table
> -
>
> Key: CASSANDRA-12041
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12041
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Carl Yeksigian
>Assignee: Joshua McKenzie
>  Labels: client-impacting
> Fix For: 3.x
>
>
> Currently we do not output CDC with {{DESCRIBE TABLE}}, but should include 
> that for 3.8+ tables.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-12081) CommitLogStressTest failing post-CASSANDRA-8844

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-12081:
--
Fix Version/s: (was: 3.8)
   3.x

> CommitLogStressTest failing post-CASSANDRA-8844
> ---
>
> Key: CASSANDRA-12081
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12081
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
> Fix For: 3.x
>
> Attachments: 0001-Fix-CommitLogStressTest.patch
>
>
> Test timing out after CASSANDRA-8844.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11734) Enable partition component index for SASI

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11734:
--
Fix Version/s: (was: 3.8)
   3.x

> Enable partition component index for SASI
> -
>
> Key: CASSANDRA-11734
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11734
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
>  Labels: doc-impacting, sasi, secondaryIndex
> Fix For: 3.x
>
> Attachments: patch.txt
>
>
> Enable partition component index for SASI
> For the given schema:
> {code:sql}
> CREATE TABLE test.comp (
> pk1 int,
> pk2 text,
> val text,
> PRIMARY KEY ((pk1, pk2))
> );
> CREATE CUSTOM INDEX comp_val_idx ON test.comp (val) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> CREATE CUSTOM INDEX comp_pk2_idx ON test.comp (pk2) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex' WITH OPTIONS = {'mode': 'PREFIX', 
> 'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.NonTokenizingAnalyzer', 
> 'case_sensitive': 'false'};
> CREATE CUSTOM INDEX comp_pk1_idx ON test.comp (pk1) USING 
> 'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> The following queries are possible:
> {code:sql}
> SELECT * FROM test.comp WHERE pk1=1;
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5;
> SELECT * FROM test.comp WHERE pk1=1 AND val='xxx' ALLOW FILTERING;
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5 AND val='xxx' ALLOW FILTERING;
> SELECT * FROM test.comp WHERE pk2='some text';
> SELECT * FROM test.comp WHERE pk2 LIKE 'prefix%';
> SELECT * FROM test.comp WHERE pk2='some text' AND val='xxx' ALLOW FILTERING;
> SELECT * FROM test.comp WHERE pk2 LIKE 'prefix%' AND val='xxx' ALLOW 
> FILTERING;
> //Without using SASI
> SELECT * FROM test.comp WHERE pk1 = 1 AND pk2='some text';
> SELECT * FROM test.comp WHERE pk1 IN(1,2,3) AND pk2='some text';
> SELECT * FROM test.comp WHERE pk1 = 1 AND pk2 IN ('text1','text2');
> SELECT * FROM test.comp WHERE pk1 IN(1,2,3) AND pk2 IN ('text1','text2');
> {code}
> However, the following queries *are not possible*
> {code:sql}
> SELECT * FROM test.comp WHERE pk1=1 AND pk2 LIKE 'prefix%';
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5 AND pk2 = 'some text';
> SELECT * FROM test.comp WHERE pk1>=1 AND pk1<=5 AND pk2 LIKE 'prefix%';
> {code}
> All of them are throwing the following exception
> {noformat}
> ava.lang.UnsupportedOperationException: null
>   at 
> org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$LikeRestriction.appendTo(SingleColumnRestriction.java:715)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.PartitionKeySingleRestrictionSet.values(PartitionKeySingleRestrictionSet.java:86)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getPartitionKeys(StatementRestrictions.java:585)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getSliceCommands(SelectStatement.java:473)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.getQuery(SelectStatement.java:265)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:230)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:79)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:208)
>  ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:239) 
> ~[main/:na]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:224) 
> ~[main/:na]
>   at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
>  ~[main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [main/:na]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [main/:na]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:283)
>  [netty-all-4.0.36.Final.jar:4.0.36.Final]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_45]
>

[jira] [Updated] (CASSANDRA-11733) cleanup unused tombstone collection in SSTableReversedIterator

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11733:
--
Fix Version/s: (was: 3.8)
   3.x

> cleanup unused tombstone collection in SSTableReversedIterator
> --
>
> Key: CASSANDRA-11733
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11733
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Dave Brosius
>Assignee: Dave Brosius
>Priority: Trivial
> Fix For: 3.x
>
> Attachments: remove_delete.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-10715) Allow filtering on NULL

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-10715:
--
Fix Version/s: (was: 4.0)
   4.x

> Allow filtering on NULL
> ---
>
> Key: CASSANDRA-10715
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10715
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
> Environment: C* 3.0.0 | cqlsh | C# driver 3.0.0beta2 | Windows 2012 R2
>Reporter: Kishan Karunaratne
>Assignee: Benjamin Lerer
>Priority: Minor
>  Labels: protocolv5
> Fix For: 4.x
>
> Attachments: 
> 0001-Allow-null-values-in-filtered-searches-reuse-Operato.patch
>
>
> This is an issue I first noticed through the C# driver, but I was able to 
> repro on cqlsh, leading me to believe this is a Cassandra bug.
> Given the following schema:
> {noformat}
> CREATE TABLE "TestKeySpace_4928dc892922"."coolMovies" (
> unique_movie_title text,
> movie_maker text,
> director text,
> list list,
> "mainGuy" text,
> "yearMade" int,
> PRIMARY KEY ((unique_movie_title, movie_maker), director)
> ) WITH CLUSTERING ORDER BY (director ASC)
> {noformat}
> Executing a SELECT with FILTERING on a non-PK column, using a NULL as the 
> argument:
> {noformat}
> SELECT "mainGuy", "movie_maker", "unique_movie_title", "list", "director", 
> "yearMade" FROM "coolMovies" WHERE "mainGuy" = null ALLOW FILTERING
> {noformat}
> returns a ReadFailure exception:
> {noformat}
> cqlsh:TestKeySpace_4c8f2cf8d5cc> SELECT "mainGuy", "movie_maker", 
> "unique_movie_title", "list", "director", "yearMade" FROM "coolMovies" WHERE 
> "mainGuy" = null ALLOW FILTERING;
> ←[0;1;31mTraceback (most recent call last):
>   File "C:\Users\Kishan\.ccm\repository\3.0.0\bin\\cqlsh.py", line 1216, in 
> perform_simple_statement
> result = future.result()
>   File 
> "C:\Users\Kishan\.ccm\repository\3.0.0\bin\..\lib\cassandra-driver-internal-only-3.0.0a3.post0-3f15725.zip\cassandra-driver-3.0.0a3.post0-3f15725\cassandra\cluster.py",
>  line 3118, in result
> raise self._final_exception
> ReadFailure: code=1300 [Replica(s) failed to execute read] message="Operation 
> failed - received 0 responses and 1 failures" info={'failures': 1, 
> 'received_responses': 0, 'required_responses': 1, 'cons
> istency': 'ONE'}
> ←[0m
> {noformat}
> Cassandra log shows:
> {noformat}
> WARN  [SharedPool-Worker-2] 2015-11-16 13:51:00,259 
> AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-2,10,main]: {}
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.db.filter.RowFilter$SimpleExpression.isSatisfiedBy(RowFilter.java:581)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToRow(RowFilter.java:243)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.transform.BaseRows.applyOne(BaseRows.java:95) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at org.apache.cassandra.db.transform.BaseRows.add(BaseRows.java:86) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.transform.UnfilteredRows.add(UnfilteredRows.java:21) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.transform.Transformation.add(Transformation.java:136) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.transform.Transformation.apply(Transformation.java:102)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:233)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.filter.RowFilter$CQLFilter$1IsSatisfiedFilter.applyToPartition(RowFilter.java:227)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:293)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:128)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
>  ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
> ~[apache-cassandra-3.0.0.jar:3.0.0]
>   at 
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:288) 
> ~[apache-cassa

[jira] [Created] (CASSANDRA-12096) dtest failure in consistency_test.TestAccuracy.test_simple_strategy_each_quorum_users

[Sean McCarthy (JIRA)]

Sean McCarthy created CASSANDRA-12096:
-

 Summary: dtest failure in 
consistency_test.TestAccuracy.test_simple_strategy_each_quorum_users
 Key: CASSANDRA-12096
 URL: https://issues.apache.org/jira/browse/CASSANDRA-12096
 Project: Cassandra
  Issue Type: Test
Reporter: Sean McCarthy
Assignee: DS Test Eng
 Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log, 
node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log, 
node4.log, node4_debug.log, node4_gc.log, node5.log, node5_debug.log, 
node5_gc.log

example failure:

http://cassci.datastax.com/job/trunk_novnode_dtest/407/testReport/consistency_test/TestAccuracy/test_simple_strategy_each_quorum_users

Failed on CassCI build trunk_novnode_dtest #407

{code}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
  File "/home/automaton/cassandra-dtest/tools.py", line 288, in wrapped
f(obj)
  File "/home/automaton/cassandra-dtest/consistency_test.py", line 591, in 
test_simple_strategy_each_quorum_users
self._run_test_function_in_parallel(TestAccuracy.Validation.validate_users, 
[self.nodes], [self.rf], combinations)
  File "/home/automaton/cassandra-dtest/consistency_test.py", line 543, in 
_run_test_function_in_parallel
assert False, err.message
'Error from server: code=2200 [Invalid query] message="unconfigured table users"
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)