[jira] [Comment Edited] (CASSANDRA-11882) Clustering Key with ByteBuffer size > 64k throws Assertion Error
[ https://issues.apache.org/jira/browse/CASSANDRA-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311554#comment-15311554 ] Lerh Chuan Low edited comment on CASSANDRA-11882 at 6/2/16 1:33 AM: Branimir & Sylvian, Thanks for the reviews so far. I was under the impression that {{Clustering}} itself is already a clustering column, good catch. I've retraced through the code path and I've updated the patch so that it only refuses larger than 64k values for clustering columns (i.e an INSERT with ckey1 = 32k and ckey2 = 32k will work, and memtable flushing still works in that case too :)). I've created the other issue here: [CASSANDRA-11943|https://issues.apache.org/jira/browse/CASSANDRA-11943]. I don't think I have permissions to assign it to Sylvain, feel free to let me know if there is anything unclear with the setup etc. was (Author: lerh low): Branimir & Sylvian, Thanks for the reviews so far. I was under the impression that {{Clustering}} itself is already a clustering column, good catch. I've retraced through the code path and I've updated the patch so that it only refuses larger than 64k values for clustering columns (i.e an INSERT with ckey1 = 32k and ckey2 = 32k will work, and memtable flushing still works in that case too :)). I've created the other issue here: https://issues.apache.org/jira/browse/CASSANDRA-11943. I don't think I have permissions to assign it to Sylvain, feel free to let me know if there is anything unclear with the setup etc. > Clustering Key with ByteBuffer size > 64k throws Assertion Error > > > Key: CASSANDRA-11882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11882 > Project: Cassandra > Issue Type: Bug > Components: CQL, Streaming and Messaging >Reporter: Lerh Chuan Low > Fix For: 2.1.x, 2.2.x > > Attachments: 11882-2.1.txt, 11882-2.2.txt, 11882-3.X.txt > > > Setup: > {code} > CREATE KEYSPACE Blues WITH REPLICATION = { 'class' : 'SimpleStrategy', > 'replication_factor' : 2}; > CREATE TABLE test (a text, b text, PRIMARY KEY ((a), b)) > {code} > There currently doesn't seem to be an existing check for selecting clustering > keys that are larger than 64k. So if we proceed to do the following select: > {code} > CONSISTENCY ALL; > SELECT * FROM Blues.test WHERE a = 'foo' AND b = 'something larger than 64k'; > {code} > An AssertionError is thrown in `ByteBufferUtil` with just a number and an > error message detailing 'Coordinator node timed out waiting for replica nodes > responses' . Additionally, because an error extends Throwable (it's not a > subclass of Exception), it's not caught so the connection between the > coordinator node and the other nodes which have the replicas seem to be > 'stuck' until it's restarted. Any other subsequent queries, even if it's just > SELECT where a = 'foo' and b = 'bar', will always return the Coordinator > timing out waiting for replica nodes responses'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11882) Clustering Key with ByteBuffer size > 64k throws Assertion Error
[ https://issues.apache.org/jira/browse/CASSANDRA-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311554#comment-15311554 ] Lerh Chuan Low commented on CASSANDRA-11882: Branimir & Sylvian, Thanks for the reviews so far. I was under the impression that {{Clustering}} itself is already a clustering column, good catch. I've retraced through the code path and I've updated the patch so that it only refuses larger than 64k values for clustering columns (i.e an INSERT with ckey1 = 32k and ckey2 = 32k will work, and memtable flushing still works in that case too :)). I've created the other issue here: https://issues.apache.org/jira/browse/CASSANDRA-11943. I don't think I have permissions to assign it to Sylvain, feel free to let me know if there is anything unclear with the setup etc. > Clustering Key with ByteBuffer size > 64k throws Assertion Error > > > Key: CASSANDRA-11882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11882 > Project: Cassandra > Issue Type: Bug > Components: CQL, Streaming and Messaging >Reporter: Lerh Chuan Low > Fix For: 2.1.x, 2.2.x > > Attachments: 11882-2.1.txt, 11882-2.2.txt, 11882-3.X.txt > > > Setup: > {code} > CREATE KEYSPACE Blues WITH REPLICATION = { 'class' : 'SimpleStrategy', > 'replication_factor' : 2}; > CREATE TABLE test (a text, b text, PRIMARY KEY ((a), b)) > {code} > There currently doesn't seem to be an existing check for selecting clustering > keys that are larger than 64k. So if we proceed to do the following select: > {code} > CONSISTENCY ALL; > SELECT * FROM Blues.test WHERE a = 'foo' AND b = 'something larger than 64k'; > {code} > An AssertionError is thrown in `ByteBufferUtil` with just a number and an > error message detailing 'Coordinator node timed out waiting for replica nodes > responses' . Additionally, because an error extends Throwable (it's not a > subclass of Exception), it's not caught so the connection between the > coordinator node and the other nodes which have the replicas seem to be > 'stuck' until it's restarted. Any other subsequent queries, even if it's just > SELECT where a = 'foo' and b = 'bar', will always return the Coordinator > timing out waiting for replica nodes responses'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11882) Clustering Key with ByteBuffer size > 64k throws Assertion Error
[ https://issues.apache.org/jira/browse/CASSANDRA-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lerh Chuan Low updated CASSANDRA-11882: --- Attachment: 11882-3.X.txt > Clustering Key with ByteBuffer size > 64k throws Assertion Error > > > Key: CASSANDRA-11882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11882 > Project: Cassandra > Issue Type: Bug > Components: CQL, Streaming and Messaging >Reporter: Lerh Chuan Low > Fix For: 2.1.x, 2.2.x > > Attachments: 11882-2.1.txt, 11882-2.2.txt, 11882-3.X.txt > > > Setup: > {code} > CREATE KEYSPACE Blues WITH REPLICATION = { 'class' : 'SimpleStrategy', > 'replication_factor' : 2}; > CREATE TABLE test (a text, b text, PRIMARY KEY ((a), b)) > {code} > There currently doesn't seem to be an existing check for selecting clustering > keys that are larger than 64k. So if we proceed to do the following select: > {code} > CONSISTENCY ALL; > SELECT * FROM Blues.test WHERE a = 'foo' AND b = 'something larger than 64k'; > {code} > An AssertionError is thrown in `ByteBufferUtil` with just a number and an > error message detailing 'Coordinator node timed out waiting for replica nodes > responses' . Additionally, because an error extends Throwable (it's not a > subclass of Exception), it's not caught so the connection between the > coordinator node and the other nodes which have the replicas seem to be > 'stuck' until it's restarted. Any other subsequent queries, even if it's just > SELECT where a = 'foo' and b = 'bar', will always return the Coordinator > timing out waiting for replica nodes responses'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11882) Clustering Key with ByteBuffer size > 64k throws Assertion Error
[ https://issues.apache.org/jira/browse/CASSANDRA-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lerh Chuan Low updated CASSANDRA-11882: --- Attachment: (was: 11882-3.X.txt) > Clustering Key with ByteBuffer size > 64k throws Assertion Error > > > Key: CASSANDRA-11882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11882 > Project: Cassandra > Issue Type: Bug > Components: CQL, Streaming and Messaging >Reporter: Lerh Chuan Low > Fix For: 2.1.x, 2.2.x > > Attachments: 11882-2.1.txt, 11882-2.2.txt, 11882-3.X.txt > > > Setup: > {code} > CREATE KEYSPACE Blues WITH REPLICATION = { 'class' : 'SimpleStrategy', > 'replication_factor' : 2}; > CREATE TABLE test (a text, b text, PRIMARY KEY ((a), b)) > {code} > There currently doesn't seem to be an existing check for selecting clustering > keys that are larger than 64k. So if we proceed to do the following select: > {code} > CONSISTENCY ALL; > SELECT * FROM Blues.test WHERE a = 'foo' AND b = 'something larger than 64k'; > {code} > An AssertionError is thrown in `ByteBufferUtil` with just a number and an > error message detailing 'Coordinator node timed out waiting for replica nodes > responses' . Additionally, because an error extends Throwable (it's not a > subclass of Exception), it's not caught so the connection between the > coordinator node and the other nodes which have the replicas seem to be > 'stuck' until it's restarted. Any other subsequent queries, even if it's just > SELECT where a = 'foo' and b = 'bar', will always return the Coordinator > timing out waiting for replica nodes responses'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11943) >64k Clustering Keys cannot be flushed
Lerh Chuan Low created CASSANDRA-11943: -- Summary: >64k Clustering Keys cannot be flushed Key: CASSANDRA-11943 URL: https://issues.apache.org/jira/browse/CASSANDRA-11943 Project: Cassandra Issue Type: Bug Components: Local Write-Read Paths Reporter: Lerh Chuan Low Setup: I set this up with a 2 node cluster, but I think with a 1 node cluster it would encounter the same issue. Use Cassandra 3. {code} CREATE KEYSPACE Blues WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 2}; CREATE TABLE test (a text, b text, PRIMARY KEY ((a), b)) {code} Do the following insert: {code} CONSISTENCY ALL; "INSERT INTO %s (a, b) VALUES ('foo', ?)", '') {code} Everything is fine and you can still run queries and so on, C* looks normal. But if we restart C*, it never succeeds in starting up: {code} java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError: Attempted serializing to buffer exceeded maximum of 65535 bytes: 131082 at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:50) ~[main/:na] at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:372) ~[main/:na] at org.apache.cassandra.db.commitlog.CommitLogReplayer.blockForWrites(CommitLogReplayer.java:257) ~[main/:na] at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189) ~[main/:na] at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:168) ~[main/:na] at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:312) [main/:na] at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:583) [main/:na] at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:712) [main/:na] Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError: Attempted serializing to buffer exceeded maximum of 65535 bytes: 131082 at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_40] at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_40] at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:365) ~[main/:na] ... 6 common frames omitted Caused by: java.lang.AssertionError: Attempted serializing to buffer exceeded maximum of 65535 bytes: 131082 at org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:309) ~[main/:na] at org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.serialize(StatsMetadata.java:286) ~[main/:na] at org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.serialize(StatsMetadata.java:235) ~[main/:na] at org.apache.cassandra.io.sstable.metadata.MetadataSerializer.serialize(MetadataSerializer.java:75) ~[main/:na] at org.apache.cassandra.io.sstable.format.big.BigTableWriter.writeMetadata(BigTableWriter.java:378) ~[main/:na] at org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$300(BigTableWriter.java:51) ~[main/:na] at org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:342) ~[main/:na] at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173) ~[main/:na] at org.apache.cassandra.io.sstable.format.SSTableWriter.prepareToCommit(SSTableWriter.java:280) ~[main/:na] at org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.prepareToCommit(SimpleSSTableMultiWriter.java:101) ~[main/:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1145) ~[main/:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1095) ~[main/:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_40] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_40] at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_40] {code} The same error as before can be reproduced if instead of restarting C* we call {{nodetool flush}} after the insert, it looks like while flushing Memtables and attempting to serialize {{SSTableMetadata}} it still expects CKeys less than 64k. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11767) dtest failure in upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD.rolling_upgrade_test
[ https://issues.apache.org/jira/browse/CASSANDRA-11767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308423#comment-15308423 ] Philip Thompson edited comment on CASSANDRA-11767 at 6/2/16 1:07 AM: - I am running so here: http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/126/ Just finished, sorry that took so long. I will never try that many again. was (Author: philipthompson): I am running so here: http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/126/ > dtest failure in > upgrade_tests.upgrade_through_versions_test.ProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD.rolling_upgrade_test > -- > > Key: CASSANDRA-11767 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11767 > Project: Cassandra > Issue Type: Bug >Reporter: Philip Thompson >Assignee: Stefania >Priority: Minor > Labels: dtest > Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, > node3.log, node3_debug.log > > > No cassci link, as jenkins is failing to show failures for this test, but I'm > seeing detected leaks. > Relevant log section: > {code} > ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,616 Ref.java:187 - LEAK > DETECTED: a reference > (org.apache.cassandra.utils.concurrent.Ref$State@41f74411) to class > org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Cleanup@1328587359:/mnt/tmp/dtest-X5kTWw/test/node1/data0/system/local-7ad54392bcdd35a684174e047860b377/la-12-big-Data.db > was not released before the reference was garbage collected > DEBUG [Reference-Reaper:1] 2016-05-11 16:17:47,617 FileCacheService.java:177 > - Invalidating cache for > /mnt/tmp/dtest-X5kTWw/test/node1/data2/system/local-7ad54392bcdd35a684174e047860b377/tmplink-la-13-big-Data.db > ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK > DETECTED: a reference > (org.apache.cassandra.utils.concurrent.Ref$State@4b0f2f61) to class > org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Cleanup@235776100:/mnt/tmp/dtest-X5kTWw/test/node1/data2/system/local-7ad54392bcdd35a684174e047860b377/tmplink-la-13-big-Data.db > was not released before the reference was garbage collected > ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK > DETECTED: a reference > (org.apache.cassandra.utils.concurrent.Ref$State@75a1b278) to class > org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1061799113:[Memory@[0..8), > Memory@[0..50)] was not released before the reference was garbage collected > ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK > DETECTED: a reference > (org.apache.cassandra.utils.concurrent.Ref$State@d0f8066) to class > org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1847744299:/mnt/tmp/dtest-X5kTWw/test/node1/data2/system/local-7ad54392bcdd35a684174e047860b377/tmplink-la-13-big-Index.db > was not released before the reference was garbage collected > ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,617 Ref.java:187 - LEAK > DETECTED: a reference > (org.apache.cassandra.utils.concurrent.Ref$State@16680ae5) to class > org.apache.cassandra.io.util.MmappedSegmentedFile$Cleanup@1744239469:/mnt/tmp/dtest-X5kTWw/test/node1/data0/system/local-7ad54392bcdd35a684174e047860b377/la-12-big-Index.db > was not released before the reference was garbage collected > ERROR [Reference-Reaper:1] 2016-05-11 16:17:47,618 Ref.java:187 - LEAK > DETECTED: a reference > (org.apache.cassandra.utils.concurrent.Ref$State@160b4b45) to class > org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@1785515424:[[OffHeapBitSet]] > was not released before the reference was garbage collected > {code} > Logs are attached. node1 is the one experiencing the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8523) Writes should be sent to a replacement node while it is streaming in data
[ https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311526#comment-15311526 ] Richard Low commented on CASSANDRA-8523: Without understanding the FD details, this sounds good. Losing hints isn't an issue, as you say. > Writes should be sent to a replacement node while it is streaming in data > - > > Key: CASSANDRA-8523 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8523 > Project: Cassandra > Issue Type: Improvement >Reporter: Richard Wagner >Assignee: Paulo Motta > Fix For: 2.1.x > > > In our operations, we make heavy use of replace_address (or > replace_address_first_boot) in order to replace broken nodes. We now realize > that writes are not sent to the replacement nodes while they are in hibernate > state and streaming in data. This runs counter to what our expectations were, > especially since we know that writes ARE sent to nodes when they are > bootstrapped into the ring. > It seems like cassandra should arrange to send writes to a node that is in > the process of replacing another node, just like it does for a nodes that are > bootstraping. I hesitate to phrase this as "we should send writes to a node > in hibernate" because the concept of hibernate may be useful in other > contexts, as per CASSANDRA-8336. Maybe a new state is needed here? > Among other things, the fact that we don't get writes during this period > makes subsequent repairs more expensive, proportional to the number of writes > that we miss (and depending on the amount of data that needs to be streamed > during replacement and the time it may take to rebuild secondary indexes, we > could miss many many hours worth of writes). It also leaves us more exposed > to consistency violations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[Cassandra Wiki] Trivial Update of "ContributorsGroup" by DaveBrosius
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ContributorsGroup" page has been changed by DaveBrosius: https://wiki.apache.org/cassandra/ContributorsGroup?action=diff=58=59 * LukasWingerberg * LyubenTodorov * JonHaddad + * MahdiMohammadi * MakiWatanabe * MarcusEriksson * MarkWatson
[jira] [Created] (CASSANDRA-11942) Cannot process role related query just after restart
Petr Malik created CASSANDRA-11942: -- Summary: Cannot process role related query just after restart Key: CASSANDRA-11942 URL: https://issues.apache.org/jira/browse/CASSANDRA-11942 Project: Cassandra Issue Type: Bug Environment: Ubuntu 14.04.4 Cassandra 3.0.6 (single node) Python (2.7) connector with Native protocol v3 Reporter: Petr Malik I get the following error from Python client when executing ALTER USER 'foo' WITH PASSWORD %s; just after service restart. It works if I wait for some 5s before executing the statement. >From system.log: 2016-06-01 22:07:01.458 InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot process role related query as the role manager isn't yet setup. This is likely because some of nodes in the cluster are on version 2.1 or earlier. You need to upgrade all nodes to Cassandra 2.2 or more to use roles." INFO [main] 2016-06-01 22:06:51,637 Server.java:162 - Starting listening for CQL clients on /127.0.0.1:9042 (unencrypted)... WARN [main] 2016-06-01 22:06:54,646 Slf4JLogger.java:136 - Failed to generate a seed from SecureRandom within 3 seconds. Not enough entrophy? INFO [main] 2016-06-01 22:06:54,680 CassandraDaemon.java:471 - Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11930) Range tombstone serialisation between 2.1 and 3.0 nodes (in 3.0 -> 2.1 direction) is broken for some Thrift deletions
[ https://issues.apache.org/jira/browse/CASSANDRA-11930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311129#comment-15311129 ] Tyler Hobbs commented on CASSANDRA-11930: - While reproducing this, I also noticed that 3.0 is returning mixed static/dynamic columns in a different order than 2.2 (when responding to a Thrift slice query). It looks like the static columns are moved to the front of the partition, like CQL stores them. Not a high priority issue to fix, but I'm mentioning it here to remind myself to open a ticket. > Range tombstone serialisation between 2.1 and 3.0 nodes (in 3.0 -> 2.1 > direction) is broken for some Thrift deletions > - > > Key: CASSANDRA-11930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11930 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Aleksey Yeschenko >Assignee: Tyler Hobbs > Fix For: 3.0.x, 3.x > > > {{LegacyLayout.LegacyRangeTombstoneList::serialize()}} has a broken > assumption that a range tombstone implies {{CompositeType}}. This is > incorrect, as you can have non-{{CompositeType}} range tombstones created via > Thrift in 2.1, and as such wrapping {{clusteringComparator.subtypes()}} in a > {{CompositeType}} is incorrect. On 2.1/2.2 side, when decoding the range > tombstone list, {{RangeTombstoneList::deserialize()}} will use the raw type > to decode start and end bounds, which will end up being confused by the extra > 2 bytes in the beginning (short length) plus end of component header in the > end. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11941) Add text option for cassandra cassandra-stress
Jose Martinez Poblete created CASSANDRA-11941: - Summary: Add text option for cassandra cassandra-stress Key: CASSANDRA-11941 URL: https://issues.apache.org/jira/browse/CASSANDRA-11941 Project: Cassandra Issue Type: Improvement Components: Tools Environment: C* 2.1.13 Reporter: Jose Martinez Poblete Currently, we are able to specify a fixed length of a text field on a YAML file as follows: {noformat} - name: text_column size: fixed(100) {noformat} That would fill our column with random characters {noformat} cqlsh:stresscql> select text_column from stresscql.msgindex limit 2; text_column --- D[\x04C[HA([o\rVae$-\x02wfC$\x00X)U\x11\x15o,zEG\\tsw)\x0b-}c\4\x15D\x0f\x1e{h[y7\x11(DIL\x12*\x01\x1fU:bRN:_T\x10\x7feN;NS\x19j?>K.q\x01dcB\x00t-nj!3;zsM1y ITb\x1bC4\x14>\x18R8\x14>M\x027|Oh\x007?\n\x164N'|ox9mBFM3\x16hq\x06}K.\x1aZM4MG$\r7X"\x0c\t\x1fX~Z3\x04~Q\x17$\x0eB4[xUc.X\x0e\x1fQ?:\x7fa\x0bl\x0b\x11Ug\x12TKP-;gv#)\F (2 rows) {noformat} For some test cases, that would be OK But for some other cases we would like to have the option to take words from a source like /usr/share/dict/words to make up for the content up to - or slightly less - the amount in bytes specified for the column for testing data models that require actual words separated by a space -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11838) dtest failure in largecolumn_test:TestLargeColumn.cleanup_test
[ https://issues.apache.org/jira/browse/CASSANDRA-11838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-11838: --- Resolution: Fixed Fix Version/s: (was: 3.x) 3.8 Status: Resolved (was: Patch Available) Committed thanks! > dtest failure in largecolumn_test:TestLargeColumn.cleanup_test > -- > > Key: CASSANDRA-11838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11838 > Project: Cassandra > Issue Type: Bug >Reporter: Philip Thompson >Assignee: Alex Petrov > Labels: dtest > Fix For: 3.8 > > Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log > > > Example failure at: > http://cassci.datastax.com/job/trunk_offheap_dtest/200/testReport/largecolumn_test/TestLargeColumn/cleanup_test/ > node 1 contains the following OOM in its log: > {code} > ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,112 Message.java:611 - > Unexpected exception during request; channel = [id: 0xb97f2640, > L:/127.0.0.1:9042 - R:/127.0.0.1:48190] > java.lang.OutOfMemoryError: Java heap space > at org.apache.cassandra.transport.CBUtil.readRawBytes(CBUtil.java:533) > ~[main/:na] > at > org.apache.cassandra.transport.CBUtil.readBoundValue(CBUtil.java:407) > ~[main/:na] > at org.apache.cassandra.transport.CBUtil.readValueList(CBUtil.java:462) > ~[main/:na] > at > org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:417) > ~[main/:na] > at > org.apache.cassandra.cql3.QueryOptions$Codec.decode(QueryOptions.java:365) > ~[main/:na] > at > org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:45) > ~[main/:na] > at > org.apache.cassandra.transport.messages.ExecuteMessage$1.decode(ExecuteMessage.java:41) > ~[main/:na] > at > org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:280) > ~[main/:na] > at > org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:261) > ~[main/:na] > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:277) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:264) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:962) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:879) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:360) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:276) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at > io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > ~[netty-all-4.0.36.Final.jar:4.0.36.Final] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] > ERROR [SharedPool-Worker-1] 2016-05-16 22:54:10,756 Message.java:611 - > Unexpected exception during request; channel = [id: 0xba0be401, > L:/127.0.0.1:9042 - R:/127.0.0.1:48191]
cassandra git commit: Cleanup recycled BTree references, trim large recycled buffers
Repository: cassandra Updated Branches: refs/heads/trunk 59949ac38 -> eb5a59a31 Cleanup recycled BTree references, trim large recycled buffers Patch by Alex Petrov; reviewed by Jake Luciani for 11838-trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eb5a59a3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eb5a59a3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eb5a59a3 Branch: refs/heads/trunk Commit: eb5a59a311a14cc0a3c37a13d10abc8c5a0f6d1b Parents: 59949ac Author: Alex PetrovAuthored: Mon May 30 09:09:41 2016 +0200 Committer: T Jake Luciani Committed: Wed Jun 1 16:24:07 2016 -0400 -- .../cassandra/io/util/DataOutputBuffer.java | 18 +++--- .../org/apache/cassandra/utils/btree/BTree.java | 1 + 2 files changed, 16 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb5a59a3/src/java/org/apache/cassandra/io/util/DataOutputBuffer.java -- diff --git a/src/java/org/apache/cassandra/io/util/DataOutputBuffer.java b/src/java/org/apache/cassandra/io/util/DataOutputBuffer.java index 8dbad8c..f4f50b1 100644 --- a/src/java/org/apache/cassandra/io/util/DataOutputBuffer.java +++ b/src/java/org/apache/cassandra/io/util/DataOutputBuffer.java @@ -40,6 +40,13 @@ public class DataOutputBuffer extends BufferedDataOutputStreamPlus */ private static final long DOUBLING_THRESHOLD = Long.getLong(Config.PROPERTY_PREFIX + "DOB_DOUBLING_THRESHOLD_MB", 64); +/* + * Only recycle OutputBuffers up to 1Mb. Larger buffers will be trimmed back to this size. + */ +private static final int MAX_RECYCLE_BUFFER_SIZE = 1024 * 1024; + +private static final int DEFAULT_INITIAL_BUFFER_SIZE = 128; + public static final Recycler RECYCLER = new Recycler() { protected DataOutputBuffer newObject(Handle handle) @@ -52,12 +59,12 @@ public class DataOutputBuffer extends BufferedDataOutputStreamPlus private DataOutputBuffer(Recycler.Handle handle) { -this(128, handle); +this(DEFAULT_INITIAL_BUFFER_SIZE, handle); } public DataOutputBuffer() { -this(128); +this(DEFAULT_INITIAL_BUFFER_SIZE); } public DataOutputBuffer(int size) @@ -79,8 +86,13 @@ public class DataOutputBuffer extends BufferedDataOutputStreamPlus public void recycle() { assert handle != null; -buffer.rewind(); +// Avoid throwing away instances that are too large, trim large buffers to default size instead. +// See CASSANDRA-11838 for details. +if (buffer().capacity() > MAX_RECYCLE_BUFFER_SIZE) +buffer = ByteBuffer.allocate(DEFAULT_INITIAL_BUFFER_SIZE); + +buffer.rewind(); RECYCLER.recycle(this, handle); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/eb5a59a3/src/java/org/apache/cassandra/utils/btree/BTree.java -- diff --git a/src/java/org/apache/cassandra/utils/btree/BTree.java b/src/java/org/apache/cassandra/utils/btree/BTree.java index 4f21d26..5665869 100644 --- a/src/java/org/apache/cassandra/utils/btree/BTree.java +++ b/src/java/org/apache/cassandra/utils/btree/BTree.java @@ -838,6 +838,7 @@ public class BTree { this.comparator = comparator; quickResolver = null; +Arrays.fill(values, 0, count, null); count = 0; detected = true; auto = true;
[jira] [Updated] (CASSANDRA-11849) Potential data directory problems due to CFS getDirectories logic
[ https://issues.apache.org/jira/browse/CASSANDRA-11849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-11849: --- Resolution: Fixed Fix Version/s: 3.0.7 3.7 Status: Resolved (was: Patch Available) committed {{7eb464734b0732ab3c6cd2a5c3409085c81f95a7}} > Potential data directory problems due to CFS getDirectories logic > - > > Key: CASSANDRA-11849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11849 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Blake Eggleston > Fix For: 3.7, 3.0.7 > > > CASSANDRA-8671 added the ability to change the data directory based on the > compaction strategy. > Since nothing uses this yet we haven't hit any issues but reading the code I > see potential bugs for things like Transaction log cleanup and CFS > initialization since these all use the default {{Directories}} location from > the yaml. > * {{Directories}} is passed into CFS constructor then possibly disregarded. > * Startup checks like scrubDataDirectories are all using default Directories > locations. > * StandaloneSSTableUtil -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbf03103 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbf03103 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbf03103 Branch: refs/heads/trunk Commit: dbf03103cc72b33b7cc021589255bd492b498907 Parents: c7e086d 7eb4647 Author: T Jake LucianiAuthored: Wed Jun 1 16:00:11 2016 -0400 Committer: T Jake Luciani Committed: Wed Jun 1 16:10:51 2016 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ColumnFamilyStore.java| 2 +- src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java | 3 ++- src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java | 3 ++- 4 files changed, 6 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbf03103/CHANGES.txt -- diff --cc CHANGES.txt index 3ecbdf4,0cafa83..a54f4fd --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,8 -1,6 +1,9 @@@ -3.0.7 +3.7 + * Fix race in CompactionStrategyManager's pause/resume (CASSANDRA-11922) +Merged from 3.0: + * Fix Directories instantiations where CFS.initialDirectories should be used (CASSANDRA-11849) * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) + * Don't use static dataDirectories field in Directories instances (CASSANDRA-11647) * Fix sstables not being protected from removal during index build (CASSANDRA-11905) * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) * Remove unneeded code to repair index summaries that have http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbf03103/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 4ea883e,3264327..f004ae0 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@@ -596,9 -578,9 +596,9 @@@ public class ColumnFamilyStore implemen * Removes unnecessary files from the cf directory at startup: these include temp files, orphans, zero-length files * and compacted sstables. Files that cannot be recognized will be ignored. */ -public static void scrubDataDirectories(CFMetaData metadata) +public static void scrubDataDirectories(CFMetaData metadata) throws StartupException { - Directories directories = new Directories(metadata); + Directories directories = new Directories(metadata, initialDirectories); // clear ephemeral snapshots that were not properly cleared last session (CASSANDRA-7357) clearEphemeralSnapshots(directories); http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbf03103/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java -- diff --cc src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java index b441454,b34ca60..bfd9739 --- a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java @@@ -380,22 -379,14 +381,22 @@@ class LogTransaction extends Transactio protected void doPrepare() { } /** - * Called on startup to scan existing folders for any unfinished leftovers of - * operations that were ongoing when the process exited. Also called by the standalone - * sstableutil tool when the cleanup option is specified, @see StandaloneSSTableUtil. + * Removes any leftovers from unifinished transactions as indicated by any transaction log files that + * are found in the table directories. This means that any old sstable files for transactions that were committed, + * or any new sstable files for transactions that were aborted or still in progress, should be removed *if + * it is safe to do so*. Refer to the checks in LogFile.verify for further details on the safety checks + * before removing transaction leftovers and refer to the comments at the beginning of this file or in NEWS.txt + * for further details on transaction logs. + * + * This method is called on startup and by the standalone sstableutil tool when the cleanup option is specified, + * @see StandaloneSSTableUtil. + * + * @return true if the leftovers of all transaction logs found were removed, false otherwise. * */ -static void removeUnfinishedLeftovers(CFMetaData metadata) +static boolean removeUnfinishedLeftovers(CFMetaData metadata) { -
[2/6] cassandra git commit: fixing Directories instantiations where CFS.initialDirectories should be used
fixing Directories instantiations where CFS.initialDirectories should be used Patch by Blake Eggleston; reviewed by tjake for CASSANDRA-11849 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7eb46473 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7eb46473 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7eb46473 Branch: refs/heads/cassandra-3.7 Commit: 7eb464734b0732ab3c6cd2a5c3409085c81f95a7 Parents: 6932bbd Author: Blake EgglestonAuthored: Wed May 25 09:09:22 2016 -0700 Committer: T Jake Luciani Committed: Wed Jun 1 15:54:07 2016 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ColumnFamilyStore.java| 2 +- src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java | 3 ++- src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java | 3 ++- 4 files changed, 6 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index a59dff1..0cafa83 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.7 + * Fix Directories instantiations where CFS.initialDirectories should be used (CASSANDRA-11849) * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) * Fix sstables not being protected from removal during index build (CASSANDRA-11905) * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 7ca6d96..3264327 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -580,7 +580,7 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean */ public static void scrubDataDirectories(CFMetaData metadata) { -Directories directories = new Directories(metadata); +Directories directories = new Directories(metadata, initialDirectories); // clear ephemeral snapshots that were not properly cleared last session (CASSANDRA-7357) clearEphemeralSnapshots(directories); http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java -- diff --git a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java index ce76165..b34ca60 100644 --- a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java @@ -32,6 +32,7 @@ import org.slf4j.LoggerFactory; import org.apache.cassandra.concurrent.ScheduledExecutors; import org.apache.cassandra.config.CFMetaData; +import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SystemKeyspace; import org.apache.cassandra.db.compaction.OperationType; @@ -385,7 +386,7 @@ class LogTransaction extends Transactional.AbstractTransactional implements Tran */ static void removeUnfinishedLeftovers(CFMetaData metadata) { -removeUnfinishedLeftovers(new Directories(metadata).getCFDirectories()); +removeUnfinishedLeftovers(new Directories(metadata, ColumnFamilyStore.getInitialDirectories()).getCFDirectories()); } @VisibleForTesting http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java -- diff --git a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java index 6e2be1d..7aa07d0 100644 --- a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java +++ b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java @@ -20,6 +20,7 @@ package org.apache.cassandra.tools; import org.apache.cassandra.config.CFMetaData; import org.apache.cassandra.config.Schema; +import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.lifecycle.LifecycleTransaction; import org.apache.cassandra.utils.OutputHandler; @@ -81,7 +82,7 @@ public class StandaloneSSTableUtil private static void listFiles(Options options, CFMetaData
[6/6] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/59949ac3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/59949ac3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/59949ac3 Branch: refs/heads/trunk Commit: 59949ac38d1e2b85a66d2b8a4fec9302766f8f90 Parents: 49a2da9 dbf0310 Author: T Jake LucianiAuthored: Wed Jun 1 16:11:11 2016 -0400 Committer: T Jake Luciani Committed: Wed Jun 1 16:11:11 2016 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ColumnFamilyStore.java| 2 +- src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java | 3 ++- src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java | 3 ++- 4 files changed, 6 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/59949ac3/CHANGES.txt --
[1/6] cassandra git commit: fixing Directories instantiations where CFS.initialDirectories should be used
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 6932bbd90 -> 7eb464734 refs/heads/cassandra-3.7 c7e086d61 -> dbf03103c refs/heads/trunk 49a2da9b7 -> 59949ac38 fixing Directories instantiations where CFS.initialDirectories should be used Patch by Blake Eggleston; reviewed by tjake for CASSANDRA-11849 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7eb46473 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7eb46473 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7eb46473 Branch: refs/heads/cassandra-3.0 Commit: 7eb464734b0732ab3c6cd2a5c3409085c81f95a7 Parents: 6932bbd Author: Blake EgglestonAuthored: Wed May 25 09:09:22 2016 -0700 Committer: T Jake Luciani Committed: Wed Jun 1 15:54:07 2016 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ColumnFamilyStore.java| 2 +- src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java | 3 ++- src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java | 3 ++- 4 files changed, 6 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index a59dff1..0cafa83 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.7 + * Fix Directories instantiations where CFS.initialDirectories should be used (CASSANDRA-11849) * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) * Fix sstables not being protected from removal during index build (CASSANDRA-11905) * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 7ca6d96..3264327 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -580,7 +580,7 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean */ public static void scrubDataDirectories(CFMetaData metadata) { -Directories directories = new Directories(metadata); +Directories directories = new Directories(metadata, initialDirectories); // clear ephemeral snapshots that were not properly cleared last session (CASSANDRA-7357) clearEphemeralSnapshots(directories); http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java -- diff --git a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java index ce76165..b34ca60 100644 --- a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java @@ -32,6 +32,7 @@ import org.slf4j.LoggerFactory; import org.apache.cassandra.concurrent.ScheduledExecutors; import org.apache.cassandra.config.CFMetaData; +import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SystemKeyspace; import org.apache.cassandra.db.compaction.OperationType; @@ -385,7 +386,7 @@ class LogTransaction extends Transactional.AbstractTransactional implements Tran */ static void removeUnfinishedLeftovers(CFMetaData metadata) { -removeUnfinishedLeftovers(new Directories(metadata).getCFDirectories()); +removeUnfinishedLeftovers(new Directories(metadata, ColumnFamilyStore.getInitialDirectories()).getCFDirectories()); } @VisibleForTesting http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java -- diff --git a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java index 6e2be1d..7aa07d0 100644 --- a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java +++ b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java @@ -20,6 +20,7 @@ package org.apache.cassandra.tools; import org.apache.cassandra.config.CFMetaData; import org.apache.cassandra.config.Schema; +import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import
[3/6] cassandra git commit: fixing Directories instantiations where CFS.initialDirectories should be used
fixing Directories instantiations where CFS.initialDirectories should be used Patch by Blake Eggleston; reviewed by tjake for CASSANDRA-11849 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7eb46473 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7eb46473 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7eb46473 Branch: refs/heads/trunk Commit: 7eb464734b0732ab3c6cd2a5c3409085c81f95a7 Parents: 6932bbd Author: Blake EgglestonAuthored: Wed May 25 09:09:22 2016 -0700 Committer: T Jake Luciani Committed: Wed Jun 1 15:54:07 2016 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ColumnFamilyStore.java| 2 +- src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java | 3 ++- src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java | 3 ++- 4 files changed, 6 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index a59dff1..0cafa83 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.7 + * Fix Directories instantiations where CFS.initialDirectories should be used (CASSANDRA-11849) * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) * Fix sstables not being protected from removal during index build (CASSANDRA-11905) * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 7ca6d96..3264327 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -580,7 +580,7 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean */ public static void scrubDataDirectories(CFMetaData metadata) { -Directories directories = new Directories(metadata); +Directories directories = new Directories(metadata, initialDirectories); // clear ephemeral snapshots that were not properly cleared last session (CASSANDRA-7357) clearEphemeralSnapshots(directories); http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java -- diff --git a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java index ce76165..b34ca60 100644 --- a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java @@ -32,6 +32,7 @@ import org.slf4j.LoggerFactory; import org.apache.cassandra.concurrent.ScheduledExecutors; import org.apache.cassandra.config.CFMetaData; +import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SystemKeyspace; import org.apache.cassandra.db.compaction.OperationType; @@ -385,7 +386,7 @@ class LogTransaction extends Transactional.AbstractTransactional implements Tran */ static void removeUnfinishedLeftovers(CFMetaData metadata) { -removeUnfinishedLeftovers(new Directories(metadata).getCFDirectories()); +removeUnfinishedLeftovers(new Directories(metadata, ColumnFamilyStore.getInitialDirectories()).getCFDirectories()); } @VisibleForTesting http://git-wip-us.apache.org/repos/asf/cassandra/blob/7eb46473/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java -- diff --git a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java index 6e2be1d..7aa07d0 100644 --- a/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java +++ b/src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java @@ -20,6 +20,7 @@ package org.apache.cassandra.tools; import org.apache.cassandra.config.CFMetaData; import org.apache.cassandra.config.Schema; +import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.lifecycle.LifecycleTransaction; import org.apache.cassandra.utils.OutputHandler; @@ -81,7 +82,7 @@ public class StandaloneSSTableUtil private static void listFiles(Options options, CFMetaData metadata,
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbf03103 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbf03103 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbf03103 Branch: refs/heads/cassandra-3.7 Commit: dbf03103cc72b33b7cc021589255bd492b498907 Parents: c7e086d 7eb4647 Author: T Jake LucianiAuthored: Wed Jun 1 16:00:11 2016 -0400 Committer: T Jake Luciani Committed: Wed Jun 1 16:10:51 2016 -0400 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/ColumnFamilyStore.java| 2 +- src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java | 3 ++- src/java/org/apache/cassandra/tools/StandaloneSSTableUtil.java | 3 ++- 4 files changed, 6 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbf03103/CHANGES.txt -- diff --cc CHANGES.txt index 3ecbdf4,0cafa83..a54f4fd --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,8 -1,6 +1,9 @@@ -3.0.7 +3.7 + * Fix race in CompactionStrategyManager's pause/resume (CASSANDRA-11922) +Merged from 3.0: + * Fix Directories instantiations where CFS.initialDirectories should be used (CASSANDRA-11849) * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) + * Don't use static dataDirectories field in Directories instances (CASSANDRA-11647) * Fix sstables not being protected from removal during index build (CASSANDRA-11905) * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) * Remove unneeded code to repair index summaries that have http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbf03103/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 4ea883e,3264327..f004ae0 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@@ -596,9 -578,9 +596,9 @@@ public class ColumnFamilyStore implemen * Removes unnecessary files from the cf directory at startup: these include temp files, orphans, zero-length files * and compacted sstables. Files that cannot be recognized will be ignored. */ -public static void scrubDataDirectories(CFMetaData metadata) +public static void scrubDataDirectories(CFMetaData metadata) throws StartupException { - Directories directories = new Directories(metadata); + Directories directories = new Directories(metadata, initialDirectories); // clear ephemeral snapshots that were not properly cleared last session (CASSANDRA-7357) clearEphemeralSnapshots(directories); http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbf03103/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java -- diff --cc src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java index b441454,b34ca60..bfd9739 --- a/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java @@@ -380,22 -379,14 +381,22 @@@ class LogTransaction extends Transactio protected void doPrepare() { } /** - * Called on startup to scan existing folders for any unfinished leftovers of - * operations that were ongoing when the process exited. Also called by the standalone - * sstableutil tool when the cleanup option is specified, @see StandaloneSSTableUtil. + * Removes any leftovers from unifinished transactions as indicated by any transaction log files that + * are found in the table directories. This means that any old sstable files for transactions that were committed, + * or any new sstable files for transactions that were aborted or still in progress, should be removed *if + * it is safe to do so*. Refer to the checks in LogFile.verify for further details on the safety checks + * before removing transaction leftovers and refer to the comments at the beginning of this file or in NEWS.txt + * for further details on transaction logs. + * + * This method is called on startup and by the standalone sstableutil tool when the cleanup option is specified, + * @see StandaloneSSTableUtil. + * + * @return true if the leftovers of all transaction logs found were removed, false otherwise. * */ -static void removeUnfinishedLeftovers(CFMetaData metadata) +static boolean removeUnfinishedLeftovers(CFMetaData metadata) { -
[jira] [Created] (CASSANDRA-11940) Look into better default file_cache_size for 2.2
T Jake Luciani created CASSANDRA-11940: -- Summary: Look into better default file_cache_size for 2.2 Key: CASSANDRA-11940 URL: https://issues.apache.org/jira/browse/CASSANDRA-11940 Project: Cassandra Issue Type: Improvement Reporter: T Jake Luciani Fix For: 2.2.x CASSANDRA-8464 added support for mmapped decompression where in version <= 2.1 the reads were all decompressed in standard heap buffers. Since the usage of the file_cache is based solely on the buffer capacity we should/can make this much larger in 2.2 when the disk access mode is mmap. The downside of this cache being too small is made worse by 8464 since the buffers are mmapped/unmapped causing explicit page faults. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[ https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310954#comment-15310954 ] Benjamin Lerer commented on CASSANDRA-11935: {quote} I don't believe in doing something for CQL because SQL does it and in particular, SQL allows pretty complicated query that we will never allow in CQL and don't want, which imply some things are less justified in CQL. We also tend to favor explicitness in C*/CQL. {quote} It is clear that there should be some differences between SQL and CQL. Nevertheless, SQL has been there for a long time and there are some areas where I believe we should learn from it. It does not make sense to me to have to write some thing like {{SELECT totalPrice / CAST(numberOfItems AS BIGINT)}} when we could simply write {{SELECT totalPrice / numberOfItem}} {quote} Also, the fact we've recently added specific explicit syntax for such conversion (CASSANDRA-10310) and are now saying some of it should be implicit make me worry that we're adding stuff to the type system without a good idea of the big picture. {quote} I do not believe that CAST and implicite conversion are not compatible. Each of them has its role. There are some case where you might want explicit casting like {{SELECT AVG(CAST (numberOfItems AS DOUBLE))}} and some other cases where you want implicite conversion. The big picture for me, in this area, is close from the one of most relational databases. I have use it for years and I always found it natural. I agree that we could make it work with explicit casting only but it will not be natural for the user as he will have to understand that numbers are typed and that you cannot add a simple integer to a double without doing an explicit cast. I have not worked on it yet but I do not believe that adding implicite conversion is complicated and I do think that it will simplify the life of the C* users. Specially if they come from an SQL background. > Add support for arithmetic operators > > > Key: CASSANDRA-11935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11935 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The goal of this ticket is to add support for arithmetic operators: > * {{-}}: Change the sign of the argument > * {{+}}: Addition operator > * {{-}}: Minus operator > * {{*}}: Multiplication operator > * {{/}}: Division operator > * {{%}}: Modulo operator > This ticket we should focus on adding operator only for numeric types to keep > the scope as small as possible. Dates and string operations will be adressed > in follow up tickets. > The operation precedence should be: > # {{*}}, {{/}}, {{%}} > # {{+}}, {{-}} > Some implicit data conversion should be performed when operations are > performed on different types (e.g. double + int). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11479) BatchlogManager unit tests failing on truncate race condition
[ https://issues.apache.org/jira/browse/CASSANDRA-11479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-11479: --- Status: Patch Available (was: In Progress) Updated the 2.2 branch and created 3.0 and 3.7 patches as well. ||branch||testall||dtest|| |[11479-2.2|https://github.com/yukim/cassandra/tree/11479-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11479-2.2-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11479-2.2-dtest/lastCompletedBuild/testReport/]| |[11479-3.0|https://github.com/yukim/cassandra/tree/11479-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11479-3.0-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11479-3.0-dtest/lastCompletedBuild/testReport/]| |[11479-3.7|https://github.com/yukim/cassandra/tree/11479-3.7]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11479-3.7-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11479-3.7-dtest/lastCompletedBuild/testReport/]| > BatchlogManager unit tests failing on truncate race condition > - > > Key: CASSANDRA-11479 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11479 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Joel Knighton >Assignee: Yuki Morishita > Fix For: 2.2.x, 3.0.x, 3.x > > Attachments: > TEST-org.apache.cassandra.batchlog.BatchlogManagerTest.log > > > Example on CI > [here|http://cassci.datastax.com/job/trunk_testall/818/testReport/junit/org.apache.cassandra.batchlog/BatchlogManagerTest/testLegacyReplay_compression/]. > This seems to have only started happening relatively recently (within the > last month or two). > As far as I can tell, this is only showing up on BatchlogManagerTests purely > because it is an aggressive user of truncate. The assertion is hit in the > setUp method, so it can happen before any of the test methods. The assertion > occurs because a compaction is happening when truncate wants to discard > SSTables; trace level logs suggest that this compaction is submitted after > the pause on the CompactionStrategyManager. > This should be reproducible by running BatchlogManagerTest in a loop - it > takes up to half an hour in my experience. A trace-level log from such a run > is attached - grep for my added log message "SSTABLES COMPACTING WHEN > DISCARDING" to find when the assert is hit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11877) Add initial support to pre-3.0 sstable serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-11877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310877#comment-15310877 ] Paulo Motta commented on CASSANDRA-11877: - bq. I'm chiming in to suggest that you should consider maintaining two formats using CASSANDRA-7443 vs maintaining the LegacyLayout class inside of the bigtable format. Thanks for the suggestion [~tjake]. We want to go that route ultimately, but for simplicity instead of using a different format we will initially use the version string within BigFormat to choose which {{BigTableWriter}} implementation to pick on {{SSTableFormat.getWriterFactory}} (if current {{BigTableWriter}} or {{LegacyBigTableWriter}}), so we focus on the data conversions now and don't get distracted with other changes to support that (such as supporting multi-format reads, etc), but we will definitely want to revisit this, possibly in a separate task. > Add initial support to pre-3.0 sstable serialization > > > Key: CASSANDRA-11877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11877 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Reporter: Paulo Motta >Assignee: Kaide Mu >Priority: Minor > > In order to support writing pre-3.0 sstables, the easiest route is to port > pre-3.0 {{BigTableWriter}} to {{LegacyLayout}}, so it can be easily removed > when this support is removed in the next majorversion. > For simplicity, we should consider only the simple case in this ticket: no > range tombstones and index sampling. We can add support to those in follow-up > tickets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11663) dtest failure in upgrade_tests.storage_engine_upgrade_test.TestStorageEngineUpgrade.upgrade_with_wide_partition_test and upgrade_with_wide_partition_reversed_test
[ https://issues.apache.org/jira/browse/CASSANDRA-11663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-11663: Assignee: Yuki Morishita (was: Joshua McKenzie) > dtest failure in > upgrade_tests.storage_engine_upgrade_test.TestStorageEngineUpgrade.upgrade_with_wide_partition_test > and upgrade_with_wide_partition_reversed_test > -- > > Key: CASSANDRA-11663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11663 > Project: Cassandra > Issue Type: Bug >Reporter: Russ Hatch >Assignee: Yuki Morishita > Labels: dtest > Attachments: node1.log, node1_debug.log > > > including two tests here, look to be failing for the same reason, example > failures: > http://cassci.datastax.com/job/trunk_dtest/1152/testReport/upgrade_tests.storage_engine_upgrade_test/TestStorageEngineUpgrade/upgrade_with_wide_partition_test > http://cassci.datastax.com/job/trunk_dtest/1152/testReport/upgrade_tests.storage_engine_upgrade_test/TestStorageEngineUpgrade/upgrade_with_wide_partition_reversed_test/ > Failed on CassCI build trunk_dtest #1152 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10783) Allow literal value as parameter of UDF & UDA
[ https://issues.apache.org/jira/browse/CASSANDRA-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310834#comment-15310834 ] Sylvain Lebresne commented on CASSANDRA-10783: -- Actually, I think we should deal with this before dealing with CASSANDRA-7396 since the latter basically needs terms in selections and that's what this introduce. There is a few things that I think the patch doesn't deal properly with however: * It sets a {{null}} name and type in the {{ColumnSpecification}} of markers. That just doesn't work. The only reason unit test don't blow up is that they don't actually exercise serializing the resulting metadata but said serialization would NPE. Besides, clients need the type to know what to send. * The way the grammar is written is imo too restrictive. It doesn't allow collection literals inside function calls for instance ({{f({ 'foo' : 'bar' })}} doesn't even parse (it parses if you add a cast before the literal, but that shouldn't be required)), which feels unecesssarily restrictive. Further, I'm generally not too fan of trying to eliminate too much through the parser as the error message ends up being confusing imo. For instance, while {{SELECT 1 FROM ...}} or {{SELECT f\(?\) FROM ...}} is allowed, {{SELECT ? FROM ...}} would throw a parsing exception. Don't get me wrong, {{SELECT ? FROM ...}} is something we have to reject, but I'd rather reject it with a clear message saying that we can't infer the type and thus require a cast. * The fact we only prepare and bind the term in {{TermSelector.getOutput()}} is a bit silly, as it means we'll redo said preparation/binding work (which, with functions can involve computation) for every output row even if it's constant once bound. In general, I feel that the right place to prepare/bind the terms is in the {{Selector.Factory.newInstance()}} method (which should thus take the {{QueryOptions}}). Now, fixing those points require quite a few changes and I took a stab at it in the patch attached below. A few points worth noting on that patch: * It makes "json" and "distinct" invalid (user) function names. It was impossible for the parser to distinguish if {{ SELECT JSON (1, 2, 3) FROM ..}} was a {{SELECT JSON}} followed by a tuple-literal, or a call to a user-defined "json" function with 3 ints arguments (same for distinct). We could, alternatively, forbid literals for {{DISTINCT}} and {{JSON}}, thus forcing the parser the recognize the query above as function calls, but if anything, that feels like the inverse of what users would expect. This would also complicate the parser and that doesn't feel worth it. Allowing those as valid UDF names just feels like a mistake in the first place and I'd rather fix it now. * It makes ColumnDefinition a Selectable (instead of ColumnIdentifier), which is both needed by the changes so we can easily get the type of a Selectable (when possible) but also make sense restrospectively. This does change a few error messages by having the check if a given column exists in only one place instead of scattered around. It also switch to ByteBuffer to handle UDT fields instead of ColumnIdentifier, mostly because the latter was not imo a good idea. * We do have to special case {{COUNT(1)}} for now, but I've simplified that special casing a bit. * It reuses the tests from Robert's patch, but with some modifications to make sure, for instance, that the metadata for bind markers is properly set. * There is actually 2 commits: the first one allow things like {{SELECT 1 FROM ...}}, defaulting {{1}} to be of type {{varint}}, much like in Robert's patch. Thinking about it though, I had a slight change of heart and removed that behavior, making the query throw an error (saying it can't infer the exact type). The reason is that I'm bothered with defaulting to {{varint}} since it's partly random and not always the most convenient. So I figured, shouldn't we take some more time to think about this and left it out initially. After all, it's not really a big deal, {{SELECT 1 FROM...}} is not a very useful query and you can always do {{SELECT (int)1 FROM ...}} if you really want. But if people feel confident enough that defaulting to {{varint}} is the only sensible choice and should be allowed now, then allowing it is as easy as ignoring the 2nd commit on the branch. || [trunk|https://github.com/pcmanus/cassandra/commits/10783] || [utests|http://cassci.datastax.com/job/pcmanus-10783-testall/] || [dtests|http://cassci.datastax.com/job/pcmanus-10783-dtest/] || > Allow literal value as parameter of UDF & UDA > - > > Key: CASSANDRA-10783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10783 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: DOAN DuyHai >
[jira] [Comment Edited] (CASSANDRA-11935) Add support for arithmetic operators
[ https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310803#comment-15310803 ] Sylvain Lebresne edited comment on CASSANDRA-11935 at 6/1/16 6:19 PM: -- I don't believe in doing something for CQL because SQL does it and in particular, SQL allows pretty complicated query that we will never allow in CQL and don't want, which imply some things are less justified in CQL. We also tend to favor explicitness in C*/CQL. Also, the fact we've recently added specific *explicit* syntax for such conversion (CASSANDRA-10310) and are now saying some of it should be implicit make me worry that we're adding stuff to the type system without a good idea of the big picture. Overall, I'm not convinced that _in CQL_ there will so many case where such implicit conversion will be useful to justify adding their magic. But at the very least, I'm pretty sure we can go far enough with operators without it and so discussing their addition should be left to another ticket (besides, such implicit conversions are not really specific to operators). was (Author: slebresne): I don't believe in doing something for CQL because SQL does it and in particular, SQL allows pretty complicated query that we will never allow in CQL and don't want, which imply some things are less justified in CQL. We also tend to favor explicitness in C*/CQL. Overall, I'm not entirely that such implicit conversions and that justify some differences. We can certainly draw inspiration from SQL when it make sense, but stuffs needs to be justified *for* CQL. Among other things, we tend to favor explicitness in C*/CQL over saving a few character in typing, and I happen to think it's a good idea. Also, the fact we've recently added specific *explicit* syntax for such conversion (CASSANDRA-10310) and are now saying some of it should be implicit make me worry that we're adding stuff to the type system without a good idea of the big picture, and that worries me. Overall, I'm not convinced that _in CQL_ there will so many case where such implicit conversion will be necessary to justify adding their magic, but at the very least, I'm pretty sure we can go far enough with operators without it and so discussing their addition should be left to another ticket (besides, such implicit conversions are not really specific to operators). > Add support for arithmetic operators > > > Key: CASSANDRA-11935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11935 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The goal of this ticket is to add support for arithmetic operators: > * {{-}}: Change the sign of the argument > * {{+}}: Addition operator > * {{-}}: Minus operator > * {{*}}: Multiplication operator > * {{/}}: Division operator > * {{%}}: Modulo operator > This ticket we should focus on adding operator only for numeric types to keep > the scope as small as possible. Dates and string operations will be adressed > in follow up tickets. > The operation precedence should be: > # {{*}}, {{/}}, {{%}} > # {{+}}, {{-}} > Some implicit data conversion should be performed when operations are > performed on different types (e.g. double + int). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[ https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310803#comment-15310803 ] Sylvain Lebresne commented on CASSANDRA-11935: -- I don't believe in doing something for CQL because SQL does it and in particular, SQL allows pretty complicated query that we will never allow in CQL and don't want, which imply some things are less justified in CQL. We also tend to favor explicitness in C*/CQL. Overall, I'm not entirely that such implicit conversions and that justify some differences. We can certainly draw inspiration from SQL when it make sense, but stuffs needs to be justified *for* CQL. Among other things, we tend to favor explicitness in C*/CQL over saving a few character in typing, and I happen to think it's a good idea. Also, the fact we've recently added specific *explicit* syntax for such conversion (CASSANDRA-10310) and are now saying some of it should be implicit make me worry that we're adding stuff to the type system without a good idea of the big picture, and that worries me. Overall, I'm not convinced that _in CQL_ there will so many case where such implicit conversion will be necessary to justify adding their magic, but at the very least, I'm pretty sure we can go far enough with operators without it and so discussing their addition should be left to another ticket (besides, such implicit conversions are not really specific to operators). > Add support for arithmetic operators > > > Key: CASSANDRA-11935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11935 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The goal of this ticket is to add support for arithmetic operators: > * {{-}}: Change the sign of the argument > * {{+}}: Addition operator > * {{-}}: Minus operator > * {{*}}: Multiplication operator > * {{/}}: Division operator > * {{%}}: Modulo operator > This ticket we should focus on adding operator only for numeric types to keep > the scope as small as possible. Dates and string operations will be adressed > in follow up tickets. > The operation precedence should be: > # {{*}}, {{/}}, {{%}} > # {{+}}, {{-}} > Some implicit data conversion should be performed when operations are > performed on different types (e.g. double + int). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7461) operator functionality in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310762#comment-15310762 ] Sylvain Lebresne commented on CASSANDRA-7461: - bq. each type of operators need to be tested and documented That's fair. > operator functionality in CQL > - > > Key: CASSANDRA-7461 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7461 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Robert Stupp >Assignee: Benjamin Lerer > Labels: cql > > Intention: Allow operators in CQL > Operators could be decimal arithmetics {{+ - * /}} or boolen arithmetics {{| > & !}} or string 'arithmetics' {{+}} > {{SELECT tab.label + ' = ' + tab.value FROM foo.tab}} > {{SELECT * FROM tab WHERE tab.label + ' = ' + tab.value = 'foo = bar'}} > as well as > {{CREATE INDEX idx ON tab ( tab.tabel + '=' + tab.value )}} > or > {{CREATE INDEX idx ON tab (label) WHERE contains(tab.tabel, > 'very-important-key')}} > Operators could be mapped to UDFs like this: > {{+}} mapped to UDF {{cstarstd::oper_plus(...)}} > {{-}} mapped to UDF {{cstarstd::oper_minus(...)}} > or handled directly via {{Cql.g}} in 'special' code -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11664) Tab completion in cqlsh doesn't work for capitalized letters
[ https://issues.apache.org/jira/browse/CASSANDRA-11664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-11664: Resolution: Fixed Fix Version/s: 3.0.7 3.7 2.2.7 Status: Resolved (was: Patch Available) The tests look good, so +1, committed to 2.2 as {{b8f5c1fcecf30e096b4a36904fdc12550910c854}} and merged up to 3.0, 3.7, and trunk. Thanks! > Tab completion in cqlsh doesn't work for capitalized letters > > > Key: CASSANDRA-11664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11664 > Project: Cassandra > Issue Type: Bug >Reporter: J.B. Langston >Assignee: Mahdi Mohammadi >Priority: Minor > Labels: cqlsh > Fix For: 2.2.7, 3.7, 3.0.7 > > > Tab completion in cqlsh doesn't work for capitalized letters, either in > keyspace names or table names. Typing quotes and a corresponding capital > letter should complete the table/keyspace name and the closing quote. > {code} > cqlsh> create keyspace "Test" WITH replication = {'class': 'SimpleStrategy', > 'replication_factor': 1}; > cqlsh> use "Tes > cqlsh> use tes > cqlsh> use Test; > InvalidRequest: code=2200 [Invalid query] message="Keyspace 'test' does not > exist" > cqlsh> use "Test"; > cqlsh:Test> drop keyspace "Test" > cqlsh:Test> create table "TestTable" (a text primary key, b text); > cqlsh:Test> select * from "TestTable"; > a | b > ---+--- > (0 rows) > cqlsh:Test> select * from "Test > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c7e086d6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c7e086d6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c7e086d6 Branch: refs/heads/trunk Commit: c7e086d61161f9bf99e0672082ce423c1fbfdce1 Parents: 01847ac 6932bbd Author: Tyler HobbsAuthored: Wed Jun 1 12:39:39 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:39:39 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7e086d6/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7e086d6/pylib/cqlshlib/cql3handling.py --
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6932bbd9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6932bbd9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6932bbd9 Branch: refs/heads/cassandra-3.0 Commit: 6932bbd909547684469ab5e2ea9e49c2e1005996 Parents: f99f8a8 b8f5c1f Author: Tyler HobbsAuthored: Wed Jun 1 12:39:23 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:39:23 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6932bbd9/CHANGES.txt -- diff --cc CHANGES.txt index 07845a6,c97293d..a59dff1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,17 -1,5 +1,18 @@@ -2.2.7 +3.0.7 + * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) + * Fix sstables not being protected from removal during index build (CASSANDRA-11905) + * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) + * Remove unneeded code to repair index summaries that have + been improperly down-sampled (CASSANDRA-11127) + * Avoid WriteTimeoutExceptions during commit log replay due to materialized + view lock contention (CASSANDRA-11891) + * Prevent OOM failures on SSTable corruption, improve tests for corruption detection (CASSANDRA-9530) + * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705) + * Allow compaction strategies to disable early open (CASSANDRA-11754) + * Refactor Materialized View code (CASSANDRA-11475) + * Update Java Driver (CASSANDRA-11615) +Merged from 2.2: + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6932bbd9/pylib/cqlshlib/cql3handling.py --
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6932bbd9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6932bbd9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6932bbd9 Branch: refs/heads/cassandra-3.7 Commit: 6932bbd909547684469ab5e2ea9e49c2e1005996 Parents: f99f8a8 b8f5c1f Author: Tyler HobbsAuthored: Wed Jun 1 12:39:23 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:39:23 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6932bbd9/CHANGES.txt -- diff --cc CHANGES.txt index 07845a6,c97293d..a59dff1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,17 -1,5 +1,18 @@@ -2.2.7 +3.0.7 + * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) + * Fix sstables not being protected from removal during index build (CASSANDRA-11905) + * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) + * Remove unneeded code to repair index summaries that have + been improperly down-sampled (CASSANDRA-11127) + * Avoid WriteTimeoutExceptions during commit log replay due to materialized + view lock contention (CASSANDRA-11891) + * Prevent OOM failures on SSTable corruption, improve tests for corruption detection (CASSANDRA-9530) + * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705) + * Allow compaction strategies to disable early open (CASSANDRA-11754) + * Refactor Materialized View code (CASSANDRA-11475) + * Update Java Driver (CASSANDRA-11615) +Merged from 2.2: + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6932bbd9/pylib/cqlshlib/cql3handling.py --
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c7e086d6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c7e086d6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c7e086d6 Branch: refs/heads/cassandra-3.7 Commit: c7e086d61161f9bf99e0672082ce423c1fbfdce1 Parents: 01847ac 6932bbd Author: Tyler HobbsAuthored: Wed Jun 1 12:39:39 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:39:39 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7e086d6/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c7e086d6/pylib/cqlshlib/cql3handling.py --
[04/10] cassandra git commit: cqlsh: Fix tab completion for case-sensitive identifiers
cqlsh: Fix tab completion for case-sensitive identifiers Patch by Mahdi Mohammadi; reviewed by Tyler Hobbs for CASSANDRA-11664 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8f5c1fc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8f5c1fc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8f5c1fc Branch: refs/heads/trunk Commit: b8f5c1fcecf30e096b4a36904fdc12550910c854 Parents: 43e9d29 Author: Mahdi MohammadiAuthored: Tue May 31 11:58:06 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:38:49 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ffd5b31..c97293d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/pylib/cqlshlib/cql3handling.py -- diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py index 81e13c9..897ee16 100644 --- a/pylib/cqlshlib/cql3handling.py +++ b/pylib/cqlshlib/cql3handling.py @@ -90,6 +90,12 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet): return str(value) return "'%s'" % value.replace("'", "''") +@classmethod +def escape_name(cls, name): +if name is None: +return 'NULL' +return "'%s'" % name.replace("'", "''") + @staticmethod def dequote_name(name): name = name.strip()
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6932bbd9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6932bbd9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6932bbd9 Branch: refs/heads/trunk Commit: 6932bbd909547684469ab5e2ea9e49c2e1005996 Parents: f99f8a8 b8f5c1f Author: Tyler HobbsAuthored: Wed Jun 1 12:39:23 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:39:23 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6932bbd9/CHANGES.txt -- diff --cc CHANGES.txt index 07845a6,c97293d..a59dff1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,17 -1,5 +1,18 @@@ -2.2.7 +3.0.7 + * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912) + * Fix sstables not being protected from removal during index build (CASSANDRA-11905) + * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032) + * Remove unneeded code to repair index summaries that have + been improperly down-sampled (CASSANDRA-11127) + * Avoid WriteTimeoutExceptions during commit log replay due to materialized + view lock contention (CASSANDRA-11891) + * Prevent OOM failures on SSTable corruption, improve tests for corruption detection (CASSANDRA-9530) + * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705) + * Allow compaction strategies to disable early open (CASSANDRA-11754) + * Refactor Materialized View code (CASSANDRA-11475) + * Update Java Driver (CASSANDRA-11615) +Merged from 2.2: + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6932bbd9/pylib/cqlshlib/cql3handling.py --
[02/10] cassandra git commit: cqlsh: Fix tab completion for case-sensitive identifiers
cqlsh: Fix tab completion for case-sensitive identifiers Patch by Mahdi Mohammadi; reviewed by Tyler Hobbs for CASSANDRA-11664 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8f5c1fc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8f5c1fc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8f5c1fc Branch: refs/heads/cassandra-3.0 Commit: b8f5c1fcecf30e096b4a36904fdc12550910c854 Parents: 43e9d29 Author: Mahdi MohammadiAuthored: Tue May 31 11:58:06 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:38:49 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ffd5b31..c97293d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/pylib/cqlshlib/cql3handling.py -- diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py index 81e13c9..897ee16 100644 --- a/pylib/cqlshlib/cql3handling.py +++ b/pylib/cqlshlib/cql3handling.py @@ -90,6 +90,12 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet): return str(value) return "'%s'" % value.replace("'", "''") +@classmethod +def escape_name(cls, name): +if name is None: +return 'NULL' +return "'%s'" % name.replace("'", "''") + @staticmethod def dequote_name(name): name = name.strip()
[10/10] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49a2da9b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49a2da9b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49a2da9b Branch: refs/heads/trunk Commit: 49a2da9b73bb36f20835aab877719c16b7c505e4 Parents: 274a8e4 c7e086d Author: Tyler HobbsAuthored: Wed Jun 1 12:39:52 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:39:52 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/49a2da9b/CHANGES.txt --
[03/10] cassandra git commit: cqlsh: Fix tab completion for case-sensitive identifiers
cqlsh: Fix tab completion for case-sensitive identifiers Patch by Mahdi Mohammadi; reviewed by Tyler Hobbs for CASSANDRA-11664 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8f5c1fc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8f5c1fc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8f5c1fc Branch: refs/heads/cassandra-3.7 Commit: b8f5c1fcecf30e096b4a36904fdc12550910c854 Parents: 43e9d29 Author: Mahdi MohammadiAuthored: Tue May 31 11:58:06 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:38:49 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ffd5b31..c97293d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/pylib/cqlshlib/cql3handling.py -- diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py index 81e13c9..897ee16 100644 --- a/pylib/cqlshlib/cql3handling.py +++ b/pylib/cqlshlib/cql3handling.py @@ -90,6 +90,12 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet): return str(value) return "'%s'" % value.replace("'", "''") +@classmethod +def escape_name(cls, name): +if name is None: +return 'NULL' +return "'%s'" % name.replace("'", "''") + @staticmethod def dequote_name(name): name = name.strip()
[01/10] cassandra git commit: cqlsh: Fix tab completion for case-sensitive identifiers
Repository: cassandra Updated Branches: refs/heads/cassandra-2.2 43e9d2936 -> b8f5c1fce refs/heads/cassandra-3.0 f99f8a8c8 -> 6932bbd90 refs/heads/cassandra-3.7 01847ac51 -> c7e086d61 refs/heads/trunk 274a8e4ff -> 49a2da9b7 cqlsh: Fix tab completion for case-sensitive identifiers Patch by Mahdi Mohammadi; reviewed by Tyler Hobbs for CASSANDRA-11664 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8f5c1fc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8f5c1fc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8f5c1fc Branch: refs/heads/cassandra-2.2 Commit: b8f5c1fcecf30e096b4a36904fdc12550910c854 Parents: 43e9d29 Author: Mahdi MohammadiAuthored: Tue May 31 11:58:06 2016 -0500 Committer: Tyler Hobbs Committed: Wed Jun 1 12:38:49 2016 -0500 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 ++ 2 files changed, 7 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ffd5b31..c97293d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8f5c1fc/pylib/cqlshlib/cql3handling.py -- diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py index 81e13c9..897ee16 100644 --- a/pylib/cqlshlib/cql3handling.py +++ b/pylib/cqlshlib/cql3handling.py @@ -90,6 +90,12 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet): return str(value) return "'%s'" % value.replace("'", "''") +@classmethod +def escape_name(cls, name): +if name is None: +return 'NULL' +return "'%s'" % name.replace("'", "''") + @staticmethod def dequote_name(name): name = name.strip()
[jira] [Updated] (CASSANDRA-11939) Read and Write Latency columns are swapped in proxyhistograms vs cfhistograms
[ https://issues.apache.org/jira/browse/CASSANDRA-11939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.B. Langston updated CASSANDRA-11939: -- Description: It’s triggering my ocd that read and write latency columns are swapped in proxyhistograms vs cfhistograms. I guess the argument against changing it now is that it could screw with some peoples scripts or expectations, but it does make it hard to eyeball when you’re trying to compare local latencies vs coordinator latencies. {code} Percentile SSTables Write Latency Read LatencyPartition Size Cell Count (micros) (micros) (bytes) 50% 4.00 17.00770.00 8239 4 75% 5.00 24.00924.00 17084 17 95% 5.00 35.00 61214.00 51012 24 98% 6.00 35.00 126934.00105778 24 99% 6.00 72.00 152321.00152321 35 Min 0.00 9.00 36.0021 0 Max 6.00 86.00 263210.00 20924300 1109 Percentile Read Latency Write Latency Range Latency (micros) (micros) (micros) 50% 1331.00535.00 11864.00 75% 17084.00642.00 20501.00 95%219342.00 1331.00 20501.00 98%315852.00 2759.00 20501.00 99%379022.00 3311.00 20501.00 Min 373.00 73.00 9888.00 Max379022.00 9887.00 20501.00 {code} Ideally read and write latencies should be in the same order and the first and second columns on both so they’re directly aligned. The sstables column should be moved to the 3rd column to make way. was: It’s triggering my ocd that read and write latency columns are swapped in proxyhistograms vs cfhistograms. I guess the argument against changing it now is that it could screw with some peoples scripts or expectations, but it does make it hard to eyeball when you’re trying to compare local latencies vs coordinator latencies. {code} Percentile SSTables Write Latency Read LatencyPartition Size Cell Count (micros) (micros) (bytes) 50% 4.00 17.00770.00 8239 4 75% 5.00 24.00924.00 17084 17 95% 5.00 35.00 61214.00 51012 24 98% 6.00 35.00 126934.00105778 24 99% 6.00 72.00 152321.00152321 35 Min 0.00 9.00 36.0021 0 Max 6.00 86.00 263210.00 20924300 1109 Percentile Read Latency Write Latency Range Latency (micros) (micros) (micros) 50% 1331.00535.00 11864.00 75% 17084.00642.00 20501.00 95%219342.00 1331.00 20501.00 98%315852.00 2759.00 20501.00 99%379022.00 3311.00 20501.00 Min 373.00 73.00 9888.00 Max379022.00 9887.00 20501.00 {code} Ideally read and write latencies should be in the same order and the first and second columns on both so they’re directly comparable. The sstables column should be moved to the 3rd column to make way. > Read and Write Latency columns are swapped in proxyhistograms vs cfhistograms > - > > Key: CASSANDRA-11939 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11939 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: J.B. Langston >Priority: Minor > > It’s triggering my ocd that read and write latency columns are swapped in > proxyhistograms vs cfhistograms. I guess the argument against changing it now > is that it could screw with some peoples scripts or expectations, but it does > make it hard to eyeball when you’re trying to compare local latencies vs > coordinator latencies. > {code} > Percentile SSTables Write Latency Read
[jira] [Updated] (CASSANDRA-11939) Read and Write Latency columns are swapped in proxyhistograms vs cfhistograms
[ https://issues.apache.org/jira/browse/CASSANDRA-11939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.B. Langston updated CASSANDRA-11939: -- Description: It’s triggering my ocd that read and write latency columns are swapped in proxyhistograms vs cfhistograms. I guess the argument against changing it now is that it could screw with some peoples scripts or expectations, but it does make it hard to eyeball when you’re trying to compare local latencies vs coordinator latencies. {code} Percentile SSTables Write Latency Read LatencyPartition Size Cell Count (micros) (micros) (bytes) 50% 4.00 17.00770.00 8239 4 75% 5.00 24.00924.00 17084 17 95% 5.00 35.00 61214.00 51012 24 98% 6.00 35.00 126934.00105778 24 99% 6.00 72.00 152321.00152321 35 Min 0.00 9.00 36.0021 0 Max 6.00 86.00 263210.00 20924300 1109 Percentile Read Latency Write Latency Range Latency (micros) (micros) (micros) 50% 1331.00535.00 11864.00 75% 17084.00642.00 20501.00 95%219342.00 1331.00 20501.00 98%315852.00 2759.00 20501.00 99%379022.00 3311.00 20501.00 Min 373.00 73.00 9888.00 Max379022.00 9887.00 20501.00 {code} Ideally read and write latencies should be in the same order and the first and second columns on both so they’re directly comparable. The sstables column should be moved to the 3rd column to make way. was: It’s triggering my ocd that read and write latency columns are swapped in proxyhistograms vs cfhistograms. I guesst the argument against changing it now is that it could screw with some peoples scripts or expectations, but it does make it hard to eyeball when you’re trying to compare local latencies vs coordinator latencies. {code} Percentile SSTables Write Latency Read LatencyPartition Size Cell Count (micros) (micros) (bytes) 50% 4.00 17.00770.00 8239 4 75% 5.00 24.00924.00 17084 17 95% 5.00 35.00 61214.00 51012 24 98% 6.00 35.00 126934.00105778 24 99% 6.00 72.00 152321.00152321 35 Min 0.00 9.00 36.0021 0 Max 6.00 86.00 263210.00 20924300 1109 Percentile Read Latency Write Latency Range Latency (micros) (micros) (micros) 50% 1331.00535.00 11864.00 75% 17084.00642.00 20501.00 95%219342.00 1331.00 20501.00 98%315852.00 2759.00 20501.00 99%379022.00 3311.00 20501.00 Min 373.00 73.00 9888.00 Max379022.00 9887.00 20501.00 {code} Ideally read and write latencies should be in the same order and the first and second columns on both so they’re directly comparable. The sstables column should be moved to the 3rd column to make way. > Read and Write Latency columns are swapped in proxyhistograms vs cfhistograms > - > > Key: CASSANDRA-11939 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11939 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: J.B. Langston >Priority: Minor > > It’s triggering my ocd that read and write latency columns are swapped in > proxyhistograms vs cfhistograms. I guess the argument against changing it now > is that it could screw with some peoples scripts or expectations, but it does > make it hard to eyeball when you’re trying to compare local latencies vs > coordinator latencies. > {code} > Percentile SSTables Write Latency
[jira] [Created] (CASSANDRA-11939) Read and Write Latency columns are swapped in proxyhistograms vs cfhistograms
J.B. Langston created CASSANDRA-11939: - Summary: Read and Write Latency columns are swapped in proxyhistograms vs cfhistograms Key: CASSANDRA-11939 URL: https://issues.apache.org/jira/browse/CASSANDRA-11939 Project: Cassandra Issue Type: Bug Components: Tools Reporter: J.B. Langston Priority: Minor It’s triggering my ocd that read and write latency columns are swapped in proxyhistograms vs cfhistograms. I guesst the argument against changing it now is that it could screw with some peoples scripts or expectations, but it does make it hard to eyeball when you’re trying to compare local latencies vs coordinator latencies. {code} Percentile SSTables Write Latency Read LatencyPartition Size Cell Count (micros) (micros) (bytes) 50% 4.00 17.00770.00 8239 4 75% 5.00 24.00924.00 17084 17 95% 5.00 35.00 61214.00 51012 24 98% 6.00 35.00 126934.00105778 24 99% 6.00 72.00 152321.00152321 35 Min 0.00 9.00 36.0021 0 Max 6.00 86.00 263210.00 20924300 1109 Percentile Read Latency Write Latency Range Latency (micros) (micros) (micros) 50% 1331.00535.00 11864.00 75% 17084.00642.00 20501.00 95%219342.00 1331.00 20501.00 98%315852.00 2759.00 20501.00 99%379022.00 3311.00 20501.00 Min 373.00 73.00 9888.00 Max379022.00 9887.00 20501.00 {code} Ideally read and write latencies should be in the same order and the first and second columns on both so they’re directly comparable. The sstables column should be moved to the 3rd column to make way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11604) select on table fails after changing user defined type in map
[ https://issues.apache.org/jira/browse/CASSANDRA-11604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Knighton updated CASSANDRA-11604: -- Status: Ready to Commit (was: Patch Available) > select on table fails after changing user defined type in map > - > > Key: CASSANDRA-11604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11604 > Project: Cassandra > Issue Type: Bug >Reporter: Andreas Jaekle >Assignee: Alex Petrov > Fix For: 3.0.x, 3.x > > > in cassandra 3.5 i get the following exception when i run this cqls: > {code} > --DROP KEYSPACE bugtest ; > CREATE KEYSPACE bugtest > WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 }; > use bugtest; > CREATE TYPE tt ( > a boolean > ); > create table t1 ( > k text, > v map, > PRIMARY KEY(k) > ); > insert into t1 (k,v) values ('k2',{'mk':{a:false}}); > ALTER TYPE tt ADD b boolean; > UPDATE t1 SET v['mk'] = { b:true } WHERE k = 'k2'; > select * from t1; > {code} > the last select fails. > {code} > WARN [SharedPool-Worker-5] 2016-04-19 14:18:49,885 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-5,5,main]: {} > java.lang.AssertionError: null > at > org.apache.cassandra.db.rows.ComplexColumnData$Builder.addCell(ComplexColumnData.java:254) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:623) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:549) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:526) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:64) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289) > ~[apache-cassandra-3.5.jar:3.5] > at >
[jira] [Commented] (CASSANDRA-11604) select on table fails after changing user defined type in map
[ https://issues.apache.org/jira/browse/CASSANDRA-11604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310666#comment-15310666 ] Joel Knighton commented on CASSANDRA-11604: --- +1 - the assert doesn't seem valid to me, and looking over Tyler's changes in trunk, I can't see a reason the assertion would be valid here but not on trunk. Tests look good. > select on table fails after changing user defined type in map > - > > Key: CASSANDRA-11604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11604 > Project: Cassandra > Issue Type: Bug >Reporter: Andreas Jaekle >Assignee: Alex Petrov > Fix For: 3.0.x, 3.x > > > in cassandra 3.5 i get the following exception when i run this cqls: > {code} > --DROP KEYSPACE bugtest ; > CREATE KEYSPACE bugtest > WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 }; > use bugtest; > CREATE TYPE tt ( > a boolean > ); > create table t1 ( > k text, > v map, > PRIMARY KEY(k) > ); > insert into t1 (k,v) values ('k2',{'mk':{a:false}}); > ALTER TYPE tt ADD b boolean; > UPDATE t1 SET v['mk'] = { b:true } WHERE k = 'k2'; > select * from t1; > {code} > the last select fails. > {code} > WARN [SharedPool-Worker-5] 2016-04-19 14:18:49,885 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-5,5,main]: {} > java.lang.AssertionError: null > at > org.apache.cassandra.db.rows.ComplexColumnData$Builder.addCell(ComplexColumnData.java:254) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:623) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:549) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:526) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:64) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76) > ~[apache-cassandra-3.5.jar:3.5] > at >
[jira] [Updated] (CASSANDRA-11604) select on table fails after changing user defined type in map
[ https://issues.apache.org/jira/browse/CASSANDRA-11604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Knighton updated CASSANDRA-11604: -- Fix Version/s: 3.0.x > select on table fails after changing user defined type in map > - > > Key: CASSANDRA-11604 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11604 > Project: Cassandra > Issue Type: Bug >Reporter: Andreas Jaekle >Assignee: Alex Petrov > Fix For: 3.0.x, 3.x > > > in cassandra 3.5 i get the following exception when i run this cqls: > {code} > --DROP KEYSPACE bugtest ; > CREATE KEYSPACE bugtest > WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 }; > use bugtest; > CREATE TYPE tt ( > a boolean > ); > create table t1 ( > k text, > v map, > PRIMARY KEY(k) > ); > insert into t1 (k,v) values ('k2',{'mk':{a:false}}); > ALTER TYPE tt ADD b boolean; > UPDATE t1 SET v['mk'] = { b:true } WHERE k = 'k2'; > select * from t1; > {code} > the last select fails. > {code} > WARN [SharedPool-Worker-5] 2016-04-19 14:18:49,885 > AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-5,5,main]: {} > java.lang.AssertionError: null > at > org.apache.cassandra.db.rows.ComplexColumnData$Builder.addCell(ComplexColumnData.java:254) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:623) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:549) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:526) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:473) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator$MergeReducer.getReduced(UnfilteredRowIterators.java:437) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:217) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:64) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76) > ~[apache-cassandra-3.5.jar:3.5] > at > org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:289) > ~[apache-cassandra-3.5.jar:3.5] > at >
[jira] [Commented] (CASSANDRA-11849) Potential data directory problems due to CFS getDirectories logic
[ https://issues.apache.org/jira/browse/CASSANDRA-11849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310631#comment-15310631 ] Blake Eggleston commented on CASSANDRA-11849: - [~tjake] rebased and retested. Dtests look ok, and I'm not able to repro any of the unit test failures. > Potential data directory problems due to CFS getDirectories logic > - > > Key: CASSANDRA-11849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11849 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Blake Eggleston > > CASSANDRA-8671 added the ability to change the data directory based on the > compaction strategy. > Since nothing uses this yet we haven't hit any issues but reading the code I > see potential bugs for things like Transaction log cleanup and CFS > initialization since these all use the default {{Directories}} location from > the yaml. > * {{Directories}} is passed into CFS constructor then possibly disregarded. > * Startup checks like scrubDataDirectories are all using default Directories > locations. > * StandaloneSSTableUtil -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11933) Improve Repair performance
[ https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310628#comment-15310628 ] Joshua McKenzie commented on CASSANDRA-11933: - Go for it - assigned it to you. > Improve Repair performance > -- > > Key: CASSANDRA-11933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11933 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Cyril Scetbon >Assignee: Mahdi Mohammadi > > During a full repair on a ~ 60 nodes cluster, I've been able to see that > this stage can be significant (up to 60 percent of the whole time) : > https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997 > It's merely caused by the fact that > https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189 > calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it > takes more than 99% of the time. This call takes 600ms when there is no load > on the cluster and more if there is. So for 10k ranges, you can imagine that > it takes at least 1.5 hours just to compute ranges. > Underneath it calls > [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170] > which can get pretty inefficient ([~jbellis]'s > [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165]) > *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend > hours on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11933) Improve Repair performance
[ https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-11933: Assignee: Mahdi Mohammadi > Improve Repair performance > -- > > Key: CASSANDRA-11933 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11933 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Cyril Scetbon >Assignee: Mahdi Mohammadi > > During a full repair on a ~ 60 nodes cluster, I've been able to see that > this stage can be significant (up to 60 percent of the whole time) : > https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997 > It's merely caused by the fact that > https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189 > calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it > takes more than 99% of the time. This call takes 600ms when there is no load > on the cluster and more if there is. So for 10k ranges, you can imagine that > it takes at least 1.5 hours just to compute ranges. > Underneath it calls > [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170] > which can get pretty inefficient ([~jbellis]'s > [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165]) > *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend > hours on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11663) dtest failure in upgrade_tests.storage_engine_upgrade_test.TestStorageEngineUpgrade.upgrade_with_wide_partition_test and upgrade_with_wide_partition_reversed_test
[ https://issues.apache.org/jira/browse/CASSANDRA-11663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310608#comment-15310608 ] Yuki Morishita commented on CASSANDRA-11663: I think this is also fixed by CASSANDRA-11763. http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/25/testReport/upgrade_tests.storage_engine_upgrade_test/TestStorageEngineUpgrade/ > dtest failure in > upgrade_tests.storage_engine_upgrade_test.TestStorageEngineUpgrade.upgrade_with_wide_partition_test > and upgrade_with_wide_partition_reversed_test > -- > > Key: CASSANDRA-11663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11663 > Project: Cassandra > Issue Type: Bug >Reporter: Russ Hatch >Assignee: Joshua McKenzie > Labels: dtest > Attachments: node1.log, node1_debug.log > > > including two tests here, look to be failing for the same reason, example > failures: > http://cassci.datastax.com/job/trunk_dtest/1152/testReport/upgrade_tests.storage_engine_upgrade_test/TestStorageEngineUpgrade/upgrade_with_wide_partition_test > http://cassci.datastax.com/job/trunk_dtest/1152/testReport/upgrade_tests.storage_engine_upgrade_test/TestStorageEngineUpgrade/upgrade_with_wide_partition_reversed_test/ > Failed on CassCI build trunk_dtest #1152 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11749) CQLSH gets SSL exception following a COPY FROM
[ https://issues.apache.org/jira/browse/CASSANDRA-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-11749: Reviewer: Tyler Hobbs Sure, I can review. > CQLSH gets SSL exception following a COPY FROM > -- > > Key: CASSANDRA-11749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11749 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Stefania >Assignee: Stefania > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x > > Attachments: driver_debug.txt, stdout.txt.zip, > stdout_single_process.txt.zip > > > When running Cassandra and cqlsh with SSL, the following command occasionally > results in the exception below: > {code} > cqlsh --ssl -f kv.cql > {code} > {code} > ERROR [SharedPool-Worker-2] 2016-05-11 12:41:03,583 Message.java:538 - > Unexpected exception during request; channel = [id: 0xeb75e05d, > /127.0.0.1:51083 => /127.0.0.1:9042] > io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: bad > record MAC > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:280) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:722) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > Caused by: javax.net.ssl.SSLException: bad record MAC > at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:981) > ~[na:1.8.0_91] > at > sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) > ~[na:1.8.0_91] > at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[na:1.8.0_91] > at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:982) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:908) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:854) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > ... 10 common frames omitted > Caused by: javax.crypto.BadPaddingException: bad record MAC > at sun.security.ssl.InputRecord.decrypt(InputRecord.java:219) > ~[na:1.8.0_91] > at > sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:177) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:974) > ~[na:1.8.0_91] > ... 17 common frames omitted > {code} > where > {code} > cat kv.cql > create keyspace if not exists cvs_copy_ks with replication = {'class': > 'SimpleStrategy', 'replication_factor':1}; > create table if not exists cvs_copy_ks.kv (key int primary key, value text); > truncate cvs_copy_ks.kv; > copy cvs_copy_ks.kv (key, value) from 'kv.csv' with header='true'; > select * from cvs_copy_ks.kv; > drop keyspace cvs_copy_ks; > stefi@cuoricina:~/git/cstar/cassandra$ cat kv.c > kv.cql kv.csv > cat kv.csv > key,value > 1,'a' > 2,'b' > 3,'c' > {code} > The COPY FROM succeeds, however the following select does not. > The easiest way to reproduce this is to restart the Cassandra
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[ https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310586#comment-15310586 ] Benjamin Lerer commented on CASSANDRA-11935: {quote} Also, I actually don't think we should allow 'double + int' at all: we're a database where the size of things matter and we shouldn't be implicit in changing the size of what we deal with. {quote} I checked ([Oracle|https://docs.oracle.com/cd/B19306_01/server.102/b14200/sql_elements002.htm#g195937], [SQLServer|https://msdn.microsoft.com/en-us/library/ms191530.aspx] and [MySQL|http://dev.mysql.com/doc/refman/5.7/en/type-conversion.html]) and they all provide implicite data conversion to allow things like 'double + int'. It is not about changing the size of data on disk, it is more about facilitating the users life by providing them some way to transform their data in what they need easily. I think that there is a lot of cases where you might want to perform some operations between 2 differents numeric types. > Add support for arithmetic operators > > > Key: CASSANDRA-11935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11935 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The goal of this ticket is to add support for arithmetic operators: > * {{-}}: Change the sign of the argument > * {{+}}: Addition operator > * {{-}}: Minus operator > * {{*}}: Multiplication operator > * {{/}}: Division operator > * {{%}}: Modulo operator > This ticket we should focus on adding operator only for numeric types to keep > the scope as small as possible. Dates and string operations will be adressed > in follow up tickets. > The operation precedence should be: > # {{*}}, {{/}}, {{%}} > # {{+}}, {{-}} > Some implicit data conversion should be performed when operations are > performed on different types (e.g. double + int). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7461) operator functionality in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310543#comment-15310543 ] Benjamin Lerer commented on CASSANDRA-7461: --- {quote} Do you think there is that much difference between those to justify spitting it? {quote} I asked myself that question. On the java side I do not think it is the case but each type of operators need to be tested and documented. Some changes needs also to be done to {{CQLSH}} for auto completion. In the end I came to the conclusion that splitting the ticket will reduce the risk of forgetting stuff and make the review process easier but I might be wrong. {quote} it's that part that I would split in a separate ticket (since again, it's not specific to operators, even if operators are a good motivation for the improvement). {quote} It is definetly something that we should do. > operator functionality in CQL > - > > Key: CASSANDRA-7461 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7461 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Robert Stupp >Assignee: Benjamin Lerer > Labels: cql > > Intention: Allow operators in CQL > Operators could be decimal arithmetics {{+ - * /}} or boolen arithmetics {{| > & !}} or string 'arithmetics' {{+}} > {{SELECT tab.label + ' = ' + tab.value FROM foo.tab}} > {{SELECT * FROM tab WHERE tab.label + ' = ' + tab.value = 'foo = bar'}} > as well as > {{CREATE INDEX idx ON tab ( tab.tabel + '=' + tab.value )}} > or > {{CREATE INDEX idx ON tab (label) WHERE contains(tab.tabel, > 'very-important-key')}} > Operators could be mapped to UDFs like this: > {{+}} mapped to UDF {{cstarstd::oper_plus(...)}} > {{-}} mapped to UDF {{cstarstd::oper_minus(...)}} > or handled directly via {{Cql.g}} in 'special' code -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11877) Add initial support to pre-3.0 sstable serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-11877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310504#comment-15310504 ] T Jake Luciani edited comment on CASSANDRA-11877 at 6/1/16 3:57 PM: I'm chiming in to suggest that you should consider maintaining two formats using CASSANDRA-7443 vs maintaining the LegacyLayout class inside of the bigtable format. This does make it hard to do though since it this approach was not used in 8099 so the "big" format marker was used in 2.2 and 3.0. You could work around this by first releasing a new format marker as part of a new release in 3.0 for the new format and use "big" only for the old format marker. was (Author: tjake): I'm chiming in to suggest that you should consider maintaining two formats using CASSANDRA-7443 vs maintaining the LegacyLayout class inside of the bigtable format. This does make it hard to do though since it this approach was not used in 8099 so the "big" format marker was used in 2.2 and 3.0 but you could start by first releasing a new format marker as part of a new release in 3.0 for the new format and use "big" only for the old format marker. > Add initial support to pre-3.0 sstable serialization > > > Key: CASSANDRA-11877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11877 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Reporter: Paulo Motta >Assignee: Kaide Mu >Priority: Minor > > In order to support writing pre-3.0 sstables, the easiest route is to port > pre-3.0 {{BigTableWriter}} to {{LegacyLayout}}, so it can be easily removed > when this support is removed in the next majorversion. > For simplicity, we should consider only the simple case in this ticket: no > range tombstones and index sampling. We can add support to those in follow-up > tickets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11877) Add initial support to pre-3.0 sstable serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-11877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310504#comment-15310504 ] T Jake Luciani commented on CASSANDRA-11877: I'm chiming in to suggest that you should consider maintaining two formats using CASSANDRA-7443 vs maintaining the LegacyLayout class inside of the bigtable format. This does make it hard to do though since it this approach was not used in 8099 so the "big" format marker was used in 2.2 and 3.0 but you could start by first releasing a new format marker as part of a new release in 3.0 for the new format and use "big" only for the old format marker. > Add initial support to pre-3.0 sstable serialization > > > Key: CASSANDRA-11877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11877 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Reporter: Paulo Motta >Assignee: Kaide Mu >Priority: Minor > > In order to support writing pre-3.0 sstables, the easiest route is to port > pre-3.0 {{BigTableWriter}} to {{LegacyLayout}}, so it can be easily removed > when this support is removed in the next majorversion. > For simplicity, we should consider only the simple case in this ticket: no > range tombstones and index sampling. We can add support to those in follow-up > tickets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11877) Add initial support to pre-3.0 sstable serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-11877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-11877: Description: In order to support writing pre-3.0 sstables, the easiest route is to port pre-3.0 {{BigTableWriter}} to {{LegacyLayout}}, so it can be easily removed when this support is removed in the next majorversion. For simplicity, we should consider only the simple case in this ticket: no range tombstones and index sampling. We can add support to those in follow-up tickets. was:In order to support writing pre-3.0 sstables, we must add support to legacy cell serialization to {{BigTableWriter}}. Summary: Add initial support to pre-3.0 sstable serialization (was: Add support to legacy row serialization on BigTableWriter) > Add initial support to pre-3.0 sstable serialization > > > Key: CASSANDRA-11877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11877 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Reporter: Paulo Motta >Assignee: Kaide Mu >Priority: Minor > > In order to support writing pre-3.0 sstables, the easiest route is to port > pre-3.0 {{BigTableWriter}} to {{LegacyLayout}}, so it can be easily removed > when this support is removed in the next majorversion. > For simplicity, we should consider only the simple case in this ticket: no > range tombstones and index sampling. We can add support to those in follow-up > tickets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11877) Add support to legacy row serialization on BigTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-11877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310446#comment-15310446 ] Paulo Motta commented on CASSANDRA-11877: - Thanks for the feedback. I definitely agree it doesn't make sense to make the 2 paradigms interoperable and it's better to keep legacy code isolated since it will probably be removed in the next major release. Since there quite a few special cases to consider (range tombstones, index sampling) let's focus on the simple case first (simple cells, no index sampling) so we can leverage existing code while having visible progress and create a basic test structure to build on top when dealing with more complex cases (range tombstones, collections, index sampling, large partitions, etc) and do the necessary improvements/optimizations later. I will update the ticket description to reflect that. I think we can start by: * Adding support to simple {{RowIndexEntry}} serialization (only position) on {{LegacyShallowIndexedEntry.serialize}} * Create {{LegacyLayout.LegacyBigTableWriter}}, which basically copies 2.2 BigTableWriter while working with the new {{SSTableWriter}} interface ({{append(UnfilteredRowIterator iterator)}}): ** Port other necessary class: {{LegacyLayout.LegacyColumnIndex}}, {{LegacyLayout.MetadataCollector}}, trying to use legacy classes from {{LegacyLayout}} where applicable ({{LegacyAtom, LegacyDeletionInfo, LegacyUnfilteredPartition}}), or classes that haven't changed between two versions ({{EstimatedHistogram, DeletionTime}} for example). ** Since we're not dealing with rangetombstones in this initial version, we can create an empty stub for {{LegacyRangeTombstoneTracker}} and port that later when dealing with range tombstones. ** Similarly, since we're not dealing with complex index columns, we can comment out parts constructing {{IndexInfo}} and always return {{ColumnIndex.EMPTY}} on {{ColumnIndex.Builder.build()}} * After the bulk structure of {{LegacyBigTableWriter}} is ported, we can probably reuse {{LegacyLayout.fromUnfilteredRowIterator}} to convert from {{UnfilteredRowIterator}} to {{LegacyUnfilteredPartition}} and work from there on {{LegacyBigTableWriter}} ** At this initial stage, since we're not dealing with range tombstones, we can probably extract the cell serialization code of {{LegacyLayout.serializeAsLegacyPartition}} to perform disk cell serialization on {{LegacyColumnIndex}} * After we have an initial draft of {{LegacyBigTableWriter}} ready, we can probably instantiate that when {{!version.storeRows}} on {{BigFormat.WriterFactory}} * Adding a few simple tests to guide the development would probably be handy, maybe we can start by making {{SimpleQuery.testTableWithoutClustering}} work with a converted sstable. [~thobbs] Does this sound better to start with and like it's going to work (even if maybe not efficiently)? Any other particular caveat we are missing or should be aware of? Thanks in advance for the help! > Add support to legacy row serialization on BigTableWriter > - > > Key: CASSANDRA-11877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11877 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Reporter: Paulo Motta >Assignee: Kaide Mu >Priority: Minor > > In order to support writing pre-3.0 sstables, we must add support to legacy > cell serialization to {{BigTableWriter}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9126) java.lang.RuntimeException: Last written key DecoratedKey >= current key DecoratedKey
[ https://issues.apache.org/jira/browse/CASSANDRA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310458#comment-15310458 ] jean carlo rivera ura commented on CASSANDRA-9126: -- hello [~mambocab]I found the ticket that correspond to our problem https://issues.apache.org/jira/browse/CASSANDRA-9935 Thank you I think we can close this ticket :) > java.lang.RuntimeException: Last written key DecoratedKey >= current key > DecoratedKey > - > > Key: CASSANDRA-9126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9126 > Project: Cassandra > Issue Type: Bug >Reporter: srinivasu gottipati >Priority: Critical > Attachments: cassandra-system.log > > > Cassandra V: 2.0.14, > Getting the following exceptions while trying to compact (I see this issue > was raised in earlier versions and marked as closed. However it still appears > in 2.0.14). In our case, compaction is not getting succeeded and keep failing > with this error.: > {code}java.lang.RuntimeException: Last written key > DecoratedKey(3462767860784856708, > 354038323137333038305f3330325f31355f474d4543454f) >= current key > DecoratedKey(3462334604624154281, > 354036333036353334315f3336315f31355f474d4543454f) writing into {code} > ... > Stacktrace:{code} > at > org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143) > at > org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166) > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) > at > org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745){code} > Any help is greatly appreciated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11938) Clarify difference between cast and cast in CQL
Sylvain Lebresne created CASSANDRA-11938: Summary: Clarify difference between cast and cast in CQL Key: CASSANDRA-11938 URL: https://issues.apache.org/jira/browse/CASSANDRA-11938 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne We have, in CQL, 2 notons of casts: * {{(int)?}} * (CAST c as int) which is pretty confusing, especially in the code where both are called "cast". Turns out both have somewhat different behavior (whether that justifies having the 2 is debatable, but it's the way it is now): * {{(int)?}} is really just a type hint for the type system. It will never "convert" value. In particular, doing {{(int)x}} where {{x}} is explicitly a {{bigint}} is invalid because {{bigint}} is 8 bytes while {{int}} is only 4 bytes. Meaning that this type of "cast" is useful when the type system needs help,for instance when you have a bind marker and the type system can't infer it's type from context. * {{(CAST c as int)}} is actually about conversions and you can do it even if {{c}} is a {{bigint}}. Now, even if both notions are useful, we can't call both "casts" or that's really confusing. As we can't really rename the 2nd form since it explicitly use {{CAST}} in the syntax, I suggest calling the first form a "type hint" (which is kind of what it is). That ticket is about reflecting that naming change in the code as well as where appropriate in the doc (we can use the occasion to improve the doc on the difference while at it). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11886) Streaming will miss sections for early opened sstables during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310435#comment-15310435 ] Benedict edited comment on CASSANDRA-11886 at 6/1/16 2:59 PM: -- If running that test, don't forget to crank up the cluster size and use vnodes, as that will have a large impact. A cluster with 100 nodes, old skool vnodes and 10k sstables would have 256M iterations (assuming we don't prune ones that cannot overlap with us, which I hope we do, but wouldn't assume) was (Author: benedict): If running that test, don't forget to crank up the cluster size and use vnodes, as that will have a large impact. A cluster with 100 nodes, old skool vnodes and 10k sstables would have 256M iterations. > Streaming will miss sections for early opened sstables during compaction > > > Key: CASSANDRA-11886 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11886 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Podkowinski >Assignee: Marcus Eriksson >Priority: Critical > Labels: correctness, repair, streaming > Attachments: 9700-test-2_1.patch > > > Once validation compaction has been finished, all mismatching sstable > sections for a token range will be used for streaming as return by > {{StreamSession.getSSTableSectionsForRanges}}. Currently 2.1 will try to > restrict the sstable candidates by checking if they can be found in > {{CANONICAL_SSTABLES}} and will ignore them otherwise. At the same time > {{IntervalTree}} in the {{DataTracker}} will be build based on replaced > non-canonical sstables as well. In case of early opened sstables this becomes > a problem, as the tree will be update with {{OpenReason.EARLY}} replacements > that cannot be found in canonical. But whenever > {{getSSTableSectionsForRanges}} will get a early instance from the view, it > will fail to retrieve the corresponding canonical version from the map, as > the different generation will cause a hashcode mismatch. Please find a test > attached. > As a consequence not all sections for a range are streamed. In our case this > has caused deleted data to reappear, as sections holding tombstones were left > out due to this behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11886) Streaming will miss sections for early opened sstables during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310435#comment-15310435 ] Benedict commented on CASSANDRA-11886: -- If running that test, don't forget to crank up the cluster size and use vnodes, as that will have a large impact. A cluster with 100 nodes, old skool vnodes and 10k sstables would have 256M iterations. > Streaming will miss sections for early opened sstables during compaction > > > Key: CASSANDRA-11886 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11886 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Podkowinski >Assignee: Marcus Eriksson >Priority: Critical > Labels: correctness, repair, streaming > Attachments: 9700-test-2_1.patch > > > Once validation compaction has been finished, all mismatching sstable > sections for a token range will be used for streaming as return by > {{StreamSession.getSSTableSectionsForRanges}}. Currently 2.1 will try to > restrict the sstable candidates by checking if they can be found in > {{CANONICAL_SSTABLES}} and will ignore them otherwise. At the same time > {{IntervalTree}} in the {{DataTracker}} will be build based on replaced > non-canonical sstables as well. In case of early opened sstables this becomes > a problem, as the tree will be update with {{OpenReason.EARLY}} replacements > that cannot be found in canonical. But whenever > {{getSSTableSectionsForRanges}} will get a early instance from the view, it > will fail to retrieve the corresponding canonical version from the map, as > the different generation will cause a hashcode mismatch. Please find a test > attached. > As a consequence not all sections for a range are streamed. In our case this > has caused deleted data to reappear, as sections holding tombstones were left > out due to this behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11935) Add support for arithmetic operators
[ https://issues.apache.org/jira/browse/CASSANDRA-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310427#comment-15310427 ] Sylvain Lebresne commented on CASSANDRA-11935: -- bq. Some implicit data conversion should be performed when operations are performed on different types (e.g. double + int) See my comment on CASSANDRA-7461. Basically, I really think we shouldn't make operators any special, they should just be syntactic sugar for normal native functions. Meaning in particular that no implicit conversion, if we add any, should be specific to operators. Which is also why I'd frame that particular discussion in another ticket. Also, I actually don't think we should allow 'double + int' at all: we're a database where the size of things matter and we shouldn't be implicit in changing the size of what we deal with. > Add support for arithmetic operators > > > Key: CASSANDRA-11935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11935 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The goal of this ticket is to add support for arithmetic operators: > * {{-}}: Change the sign of the argument > * {{+}}: Addition operator > * {{-}}: Minus operator > * {{*}}: Multiplication operator > * {{/}}: Division operator > * {{%}}: Modulo operator > This ticket we should focus on adding operator only for numeric types to keep > the scope as small as possible. Dates and string operations will be adressed > in follow up tickets. > The operation precedence should be: > # {{*}}, {{/}}, {{%}} > # {{+}}, {{-}} > Some implicit data conversion should be performed when operations are > performed on different types (e.g. double + int). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7461) operator functionality in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310417#comment-15310417 ] Sylvain Lebresne commented on CASSANDRA-7461: - Do you think there is that much difference between those to justify spitting it? I mean, I don't mind it in principle, but I feel it'll just split the discussion on what feels to me like essentially the same thing, and will create arbitrary dependencies between the patches. Overall, I strongly feel that operator should just be syntactic sugar native functions. In particular, on the grammar side, I'd let operators act between any 2 terms, and so supporting {{+}} on numbers or on strings will be the same outside of dispatching to different (trivial) methods based on the type. The main issue I see on this ticket is that our current type system is necessarily up to the task, at least not if we want a good user experience, mainly due to the fact that we'll need multiple overload of the operator functions for every type (typically a {{add(int, int)}}, {{add(bigint, bigint)}}, ) and that if we do that currently, things like {{add(2, 3)}} won't type-check because it will complain that multiple overload could apply (true) and that it can't make a choice. This problem is not really specific related to operators however, it's more about how we make functions resolution when there is overloads smarter (which I think we can do but I'll let that discussion for another comment) and if anything, it's that part that I would split in a separate ticket (since again, it's not specific to operators, even if operators are a good motivation for the improvement). > operator functionality in CQL > - > > Key: CASSANDRA-7461 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7461 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Robert Stupp >Assignee: Benjamin Lerer > Labels: cql > > Intention: Allow operators in CQL > Operators could be decimal arithmetics {{+ - * /}} or boolen arithmetics {{| > & !}} or string 'arithmetics' {{+}} > {{SELECT tab.label + ' = ' + tab.value FROM foo.tab}} > {{SELECT * FROM tab WHERE tab.label + ' = ' + tab.value = 'foo = bar'}} > as well as > {{CREATE INDEX idx ON tab ( tab.tabel + '=' + tab.value )}} > or > {{CREATE INDEX idx ON tab (label) WHERE contains(tab.tabel, > 'very-important-key')}} > Operators could be mapped to UDFs like this: > {{+}} mapped to UDF {{cstarstd::oper_plus(...)}} > {{-}} mapped to UDF {{cstarstd::oper_minus(...)}} > or handled directly via {{Cql.g}} in 'special' code -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7461) operator functionality in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310321#comment-15310321 ] Benjamin Lerer edited comment on CASSANDRA-7461 at 6/1/16 2:38 PM: --- There are 3 categories of operators: * Arithmetic operators: {{+}}, {{-}}, {{*}}, {{/}} and {{%}} * Logical operators: {{|}}, {{&}}, {{!}} and {{XOR}} * String concatenation: {{+}} I think it is probably best to use this ticket as an umbrella ticket and to split the ticket into sub tasks. was (Author: blerer): There are 3 categories of operators: * Arithmetic operators: {{+}}, {{-}}, {{*}}, {{/}} and {{%}} * Logical operators: {{|}}, {{&}}, {{!}} and {{XOR}} * String concatenation: {{+}} I think it is probably best to use this ticket as an umbrella ticket and to split the ticket into 3 sub task. > operator functionality in CQL > - > > Key: CASSANDRA-7461 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7461 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Robert Stupp >Assignee: Benjamin Lerer > Labels: cql > > Intention: Allow operators in CQL > Operators could be decimal arithmetics {{+ - * /}} or boolen arithmetics {{| > & !}} or string 'arithmetics' {{+}} > {{SELECT tab.label + ' = ' + tab.value FROM foo.tab}} > {{SELECT * FROM tab WHERE tab.label + ' = ' + tab.value = 'foo = bar'}} > as well as > {{CREATE INDEX idx ON tab ( tab.tabel + '=' + tab.value )}} > or > {{CREATE INDEX idx ON tab (label) WHERE contains(tab.tabel, > 'very-important-key')}} > Operators could be mapped to UDFs like this: > {{+}} mapped to UDF {{cstarstd::oper_plus(...)}} > {{-}} mapped to UDF {{cstarstd::oper_minus(...)}} > or handled directly via {{Cql.g}} in 'special' code -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11886) Streaming will miss sections for early opened sstables during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310396#comment-15310396 ] Marcus Eriksson commented on CASSANDRA-11886: - pushed to all branches below ||branch||testall||dtest|| |[marcuse/11886|https://github.com/krummas/cassandra/tree/marcuse/11886]|[testall|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-dtest]| |[marcuse/11886-2.2|https://github.com/krummas/cassandra/tree/marcuse/11886-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-2.2-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-2.2-dtest]| |[marcuse/11886-3.0|https://github.com/krummas/cassandra/tree/marcuse/11886-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-3.0-dtest]| |[marcuse/11886-3.7|https://github.com/krummas/cassandra/tree/marcuse/11886-3.7]|[testall|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-3.7-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-3.7-dtest]| |[marcuse/11886-trunk|https://github.com/krummas/cassandra/tree/marcuse/11886-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11886-trunk-dtest]| I'll run some benchmarks to see how bad iterating over all sstables gets if we have several thousand sstables to see if we need to do the overlap iterator now or leave that as an improvement in another ticket > Streaming will miss sections for early opened sstables during compaction > > > Key: CASSANDRA-11886 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11886 > Project: Cassandra > Issue Type: Bug >Reporter: Stefan Podkowinski >Assignee: Marcus Eriksson >Priority: Critical > Labels: correctness, repair, streaming > Attachments: 9700-test-2_1.patch > > > Once validation compaction has been finished, all mismatching sstable > sections for a token range will be used for streaming as return by > {{StreamSession.getSSTableSectionsForRanges}}. Currently 2.1 will try to > restrict the sstable candidates by checking if they can be found in > {{CANONICAL_SSTABLES}} and will ignore them otherwise. At the same time > {{IntervalTree}} in the {{DataTracker}} will be build based on replaced > non-canonical sstables as well. In case of early opened sstables this becomes > a problem, as the tree will be update with {{OpenReason.EARLY}} replacements > that cannot be found in canonical. But whenever > {{getSSTableSectionsForRanges}} will get a early instance from the view, it > will fail to retrieve the corresponding canonical version from the map, as > the different generation will cause a hashcode mismatch. Please find a test > attached. > As a consequence not all sections for a range are streamed. In our case this > has caused deleted data to reappear, as sections holding tombstones were left > out due to this behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11575) Add out-of-process testing for CDC
[ https://issues.apache.org/jira/browse/CASSANDRA-11575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-11575: --- Attachment: 11575.tgz I've attached a simple CDC daemon for running testing outside of Cassandra. To run it: - Specify the Cassandra cdc_raw directory: {{-Dcassandra.cdc_raw=/etc/cassandra/data/cdc_raw}} - Specify the Cassandra commitlog directory: {{-Dcassandra.commitlog=/etc/cassandra/data/commitlog}} - Local Cassandra running to pull the schema from Dependency is on a 3.8-SNAPSHOT cassandra, so running {{ant publish}} in the cassandra directory will put the necessary jars into Maven. > Add out-of-process testing for CDC > -- > > Key: CASSANDRA-11575 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11575 > Project: Cassandra > Issue Type: Sub-task > Components: Coordination, Local Write-Read Paths >Reporter: Carl Yeksigian >Assignee: Carl Yeksigian > Fix For: 3.x > > Attachments: 11575.tgz > > > There are currently no dtests for the new cdc feature. We should have some, > at least to ensure that the cdc files have a lifecycle that makes sense, and > make sure that things like a continually cleaning daemon and a lazy daemon > have the properties we expect; for this, we don't need to actually process > the files, but make sure they fit the characteristics we expect from them. A > more complex daemon would need to be written in Java. > I already hit a problem where if the cdc is over capacity, the cdc properly > throws the WTE, but it will not reset after the overflow directory is > undersize again. It is supposed to correct the size within 250ms and allow > more writes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade
Alex Petrov created CASSANDRA-11937: --- Summary: Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade Key: CASSANDRA-11937 URL: https://issues.apache.org/jira/browse/CASSANDRA-11937 Project: Cassandra Issue Type: Improvement Reporter: Alex Petrov In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're trimming the large buffers in {{DataOutputBuffer}}. The patch is already submitted and merged in [Netty 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa], we only need to make sure that we throw large buffers away1 alltogether instead of trimming them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-11937: Labels: lhf netty reminder (was: lhf netty) > Clean up buffer trimming large buffers in DataOutputBuffer after the Netty > upgrade > -- > > Key: CASSANDRA-11937 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11937 > Project: Cassandra > Issue Type: Improvement >Reporter: Alex Petrov > Labels: lhf, netty, reminder > > In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're > trimming the large buffers in {{DataOutputBuffer}}. The patch is already > submitted and merged in [Netty > 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa], > we only need to make sure that we throw large buffers away1 alltogether > instead of trimming them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-11937: Labels: lhf netty (was: lhf) > Clean up buffer trimming large buffers in DataOutputBuffer after the Netty > upgrade > -- > > Key: CASSANDRA-11937 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11937 > Project: Cassandra > Issue Type: Improvement >Reporter: Alex Petrov > Labels: lhf, netty, reminder > > In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're > trimming the large buffers in {{DataOutputBuffer}}. The patch is already > submitted and merged in [Netty > 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa], > we only need to make sure that we throw large buffers away1 alltogether > instead of trimming them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11936) Add support for + and - operations on dates
Benjamin Lerer created CASSANDRA-11936: -- Summary: Add support for + and - operations on dates Key: CASSANDRA-11936 URL: https://issues.apache.org/jira/browse/CASSANDRA-11936 Project: Cassandra Issue Type: Sub-task Reporter: Benjamin Lerer Assignee: Benjamin Lerer Fix For: 3.x For time series it can be interesting to allow queries with {{WHERE}} clause like: {{... WHERE reading_time < now() - 2h}} In order to do that we need to add support for: {{+}} and {{-}} operation with date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10070) Automatic repair scheduling
[ https://issues.apache.org/jira/browse/CASSANDRA-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Olsson updated CASSANDRA-10070: -- Attachment: Distributed Repair Scheduling_V2.doc I have attached an updated feature draft, if anything seems unclear just let me know! :) > Automatic repair scheduling > --- > > Key: CASSANDRA-10070 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10070 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Olsson >Assignee: Marcus Olsson >Priority: Minor > Fix For: 3.x > > Attachments: Distributed Repair Scheduling.doc, Distributed Repair > Scheduling_V2.doc > > > Scheduling and running repairs in a Cassandra cluster is most often a > required task, but this can both be hard for new users and it also requires a > bit of manual configuration. There are good tools out there that can be used > to simplify things, but wouldn't this be a good feature to have inside of > Cassandra? To automatically schedule and run repairs, so that when you start > up your cluster it basically maintains itself in terms of normal > anti-entropy, with the possibility for manual configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11935) Add support for arithmetic operators
Benjamin Lerer created CASSANDRA-11935: -- Summary: Add support for arithmetic operators Key: CASSANDRA-11935 URL: https://issues.apache.org/jira/browse/CASSANDRA-11935 Project: Cassandra Issue Type: Sub-task Reporter: Benjamin Lerer Assignee: Benjamin Lerer Fix For: 3.x The goal of this ticket is to add support for arithmetic operators: * {{-}}: Change the sign of the argument * {{+}}: Addition operator * {{-}}: Minus operator * {{*}}: Multiplication operator * {{/}}: Division operator * {{%}}: Modulo operator This ticket we should focus on adding operator only for numeric types to keep the scope as small as possible. Dates and string operations will be adressed in follow up tickets. The operation precedence should be: # {{*}}, {{/}}, {{%}} # {{+}}, {{-}} Some implicit data conversion should be performed when operations are performed on different types (e.g. double + int). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'
[ https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310357#comment-15310357 ] Stefania commented on CASSANDRA-11055: -- Thanks. > C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no > attribute 'replace' > --- > > Key: CASSANDRA-11055 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11055 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Simon Ashley >Assignee: Stefania > Labels: cqlsh > Fix For: 2.1.x > > Attachments: 11055-driver-2.7.2.patch, data.tar.gz > > > C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns: > {code} > 'NoneType' object has no attribute 'replace' > {code} > for thrift CF's originally created in C* 1.2. > Repro: > 1. Create cf in cassandra-cli on C* 1.2.x (1.2.9 was used here) > {code} > [default@ks1] CREATE COLUMN FAMILY t1 > ... WITH column_type='Standard' > ... AND > comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)' > ... AND default_validation_class='UTF8Type' > ... AND key_validation_class='UTF8Type' > ... AND read_repair_chance=0.1 > ... AND dclocal_read_repair_chance=0.0 > ... AND gc_grace=864000 > ... AND min_compaction_threshold=4 > ... AND max_compaction_threshold=32 > ... AND replicate_on_write=true > ... AND compaction_strategy='LeveledCompactionStrategy' AND > compaction_strategy_options={sstable_size_in_mb: 32} > ... AND caching='KEYS_ONLY' > ... AND compression_options={sstable_compression:SnappyCompressor, > chunk_length_kb:64}; > qlsh> describe keyspace ks1; > CREATE KEYSPACE ks1 WITH replication = { > 'class': 'NetworkTopologyStrategy', > 'datacenter1': '1' > }; > USE ks1; > CREATE TABLE t1 ( > key text, > column1 text, > column2 text, > value text, > PRIMARY KEY (key, column1, column2) > ) WITH COMPACT STORAGE AND > bloom_filter_fp_chance=0.10 AND > caching='KEYS_ONLY' AND > comment='' AND > dclocal_read_repair_chance=0.00 AND > gc_grace_seconds=864000 AND > read_repair_chance=0.10 AND > replicate_on_write='true' AND > populate_io_cache_on_flush='false' AND > compaction={'sstable_size_in_mb': '32', 'class': > 'LeveledCompactionStrategy'} AND > compression={'chunk_length_kb': '64', 'sstable_compression': > 'SnappyCompressor'}; > cqlsh> select keyspace_name, columnfamily_name,column_aliases,key_aliases > from system.schema_columnfamilies where keyspace_name= 'ks1'; > keyspace_name | columnfamily_name | column_aliases | key_aliases > ---+---++- >ks1 |t1 | [] | [] > 2/ Upgrade -> C* 2.0.9 -> nodetool upgradesstables -a > At this stage , DESCRIBE in cqlsh is working > 3/ Upgrade -> C* 2.1.12 -> nodetool upgradesstables -a > DESCRIBE now fails: > cqlsh> describe table ks1.t1; > 'NoneType' object has no attribute 'replace' > cqlsh> describe keyspace ks1; > 'NoneType' object has no attribute 'replace' > {code} > You can workaround by manually updating {{system.schema_columnfamilies}} > {code} > UPDATE system.schema_columnfamilies SET column_aliases > ='["column1","column2"]' WHERE keyspace_name = 'ks1' AND columnfamily_name = > 't1'; > {code} > Once you exit and restart cqlsh, {{DESCRIBE}} is not working as per C* 1.2 > {code} > cqlsh> describe keyspace ks1; > CREATE KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', > 'datacenter1': '1'} AND durable_writes = true; > CREATE TABLE ks1.t1 ( > key text, > column1 text, > column2 text, > value text, > PRIMARY KEY (key, column1, column2) > ) WITH COMPACT STORAGE > AND CLUSTERING ORDER BY (column1 ASC, column2 ASC) > AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' > AND comment = '' > AND compaction = {'sstable_size_in_mb': '32', 'class': > 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'} > AND compression = {'chunk_length_kb': '64', 'sstable_compression': > 'org.apache.cassandra.io.compress.SnappyCompressor'} > AND dclocal_read_repair_chance = 0.0 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.1 > AND speculative_retry = '99.0PERCENTILE'; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'
[ https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310353#comment-15310353 ] Adam Holmberg commented on CASSANDRA-11055: --- [PYTHON-579|https://datastax-oss.atlassian.net/browse/PYTHON-579] to follow up on DCT in current driver. > C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no > attribute 'replace' > --- > > Key: CASSANDRA-11055 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11055 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Simon Ashley >Assignee: Stefania > Labels: cqlsh > Fix For: 2.1.x > > Attachments: 11055-driver-2.7.2.patch, data.tar.gz > > > C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns: > {code} > 'NoneType' object has no attribute 'replace' > {code} > for thrift CF's originally created in C* 1.2. > Repro: > 1. Create cf in cassandra-cli on C* 1.2.x (1.2.9 was used here) > {code} > [default@ks1] CREATE COLUMN FAMILY t1 > ... WITH column_type='Standard' > ... AND > comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)' > ... AND default_validation_class='UTF8Type' > ... AND key_validation_class='UTF8Type' > ... AND read_repair_chance=0.1 > ... AND dclocal_read_repair_chance=0.0 > ... AND gc_grace=864000 > ... AND min_compaction_threshold=4 > ... AND max_compaction_threshold=32 > ... AND replicate_on_write=true > ... AND compaction_strategy='LeveledCompactionStrategy' AND > compaction_strategy_options={sstable_size_in_mb: 32} > ... AND caching='KEYS_ONLY' > ... AND compression_options={sstable_compression:SnappyCompressor, > chunk_length_kb:64}; > qlsh> describe keyspace ks1; > CREATE KEYSPACE ks1 WITH replication = { > 'class': 'NetworkTopologyStrategy', > 'datacenter1': '1' > }; > USE ks1; > CREATE TABLE t1 ( > key text, > column1 text, > column2 text, > value text, > PRIMARY KEY (key, column1, column2) > ) WITH COMPACT STORAGE AND > bloom_filter_fp_chance=0.10 AND > caching='KEYS_ONLY' AND > comment='' AND > dclocal_read_repair_chance=0.00 AND > gc_grace_seconds=864000 AND > read_repair_chance=0.10 AND > replicate_on_write='true' AND > populate_io_cache_on_flush='false' AND > compaction={'sstable_size_in_mb': '32', 'class': > 'LeveledCompactionStrategy'} AND > compression={'chunk_length_kb': '64', 'sstable_compression': > 'SnappyCompressor'}; > cqlsh> select keyspace_name, columnfamily_name,column_aliases,key_aliases > from system.schema_columnfamilies where keyspace_name= 'ks1'; > keyspace_name | columnfamily_name | column_aliases | key_aliases > ---+---++- >ks1 |t1 | [] | [] > 2/ Upgrade -> C* 2.0.9 -> nodetool upgradesstables -a > At this stage , DESCRIBE in cqlsh is working > 3/ Upgrade -> C* 2.1.12 -> nodetool upgradesstables -a > DESCRIBE now fails: > cqlsh> describe table ks1.t1; > 'NoneType' object has no attribute 'replace' > cqlsh> describe keyspace ks1; > 'NoneType' object has no attribute 'replace' > {code} > You can workaround by manually updating {{system.schema_columnfamilies}} > {code} > UPDATE system.schema_columnfamilies SET column_aliases > ='["column1","column2"]' WHERE keyspace_name = 'ks1' AND columnfamily_name = > 't1'; > {code} > Once you exit and restart cqlsh, {{DESCRIBE}} is not working as per C* 1.2 > {code} > cqlsh> describe keyspace ks1; > CREATE KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', > 'datacenter1': '1'} AND durable_writes = true; > CREATE TABLE ks1.t1 ( > key text, > column1 text, > column2 text, > value text, > PRIMARY KEY (key, column1, column2) > ) WITH COMPACT STORAGE > AND CLUSTERING ORDER BY (column1 ASC, column2 ASC) > AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' > AND comment = '' > AND compaction = {'sstable_size_in_mb': '32', 'class': > 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'} > AND compression = {'chunk_length_kb': '64', 'sstable_compression': > 'org.apache.cassandra.io.compress.SnappyCompressor'} > AND dclocal_read_repair_chance = 0.0 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.1 > AND speculative_retry = '99.0PERCENTILE'; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11749) CQLSH gets SSL exception following a COPY FROM
[ https://issues.apache.org/jira/browse/CASSANDRA-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310349#comment-15310349 ] Stefania commented on CASSANDRA-11749: -- I've reverted the changes for {{PlainTextAuthProvider}} since they were not necessary and were causing problems. I've restarted the tests for 2.1, if they are OK I'll restart the other branches as well. > CQLSH gets SSL exception following a COPY FROM > -- > > Key: CASSANDRA-11749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11749 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Stefania >Assignee: Stefania > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x > > Attachments: driver_debug.txt, stdout.txt.zip, > stdout_single_process.txt.zip > > > When running Cassandra and cqlsh with SSL, the following command occasionally > results in the exception below: > {code} > cqlsh --ssl -f kv.cql > {code} > {code} > ERROR [SharedPool-Worker-2] 2016-05-11 12:41:03,583 Message.java:538 - > Unexpected exception during request; channel = [id: 0xeb75e05d, > /127.0.0.1:51083 => /127.0.0.1:9042] > io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: bad > record MAC > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:280) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:722) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > Caused by: javax.net.ssl.SSLException: bad record MAC > at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:981) > ~[na:1.8.0_91] > at > sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) > ~[na:1.8.0_91] > at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[na:1.8.0_91] > at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:982) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:908) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:854) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > at > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249) > ~[netty-all-4.0.23.Final.jar:4.0.23.Final] > ... 10 common frames omitted > Caused by: javax.crypto.BadPaddingException: bad record MAC > at sun.security.ssl.InputRecord.decrypt(InputRecord.java:219) > ~[na:1.8.0_91] > at > sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:177) > ~[na:1.8.0_91] > at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:974) > ~[na:1.8.0_91] > ... 17 common frames omitted > {code} > where > {code} > cat kv.cql > create keyspace if not exists cvs_copy_ks with replication = {'class': > 'SimpleStrategy', 'replication_factor':1}; > create table if not exists cvs_copy_ks.kv (key int primary key, value text); > truncate cvs_copy_ks.kv; > copy cvs_copy_ks.kv (key, value) from 'kv.csv' with header='true'; > select * from cvs_copy_ks.kv; > drop keyspace cvs_copy_ks; > stefi@cuoricina:~/git/cstar/cassandra$ cat kv.c > kv.cql
[jira] [Commented] (CASSANDRA-11763) Failure to read non-shallow pre-3.0 sstable index entries
[ https://issues.apache.org/jira/browse/CASSANDRA-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310346#comment-15310346 ] Philip Thompson commented on CASSANDRA-11763: - JQL search fails me yet again. Then, no problems with the tests, and I no longer see the issue from this ticket [11763]. > Failure to read non-shallow pre-3.0 sstable index entries > - > > Key: CASSANDRA-11763 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11763 > Project: Cassandra > Issue Type: Bug >Reporter: Philip Thompson >Assignee: Robert Stupp > Labels: dtest > Fix For: 3.6 > > Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, > node3.log, node3_debug.log > > > example failure in: > http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/12/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV3Upgrade_AllVersions_RandomPartitioner_Skips_3_0_x_EndsAt_Trunk_HEAD/rolling_upgrade_test_2/ > Logs are attached, relevant stack trace is here. > {code} > ERROR [main] 2016-05-11 16:26:06,555 CassandraDaemon.java:727 - Exception > encountered during startup > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.NegativeArraySizeException > at > org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:119) > ~[main/:na] > at > org.apache.cassandra.hints.LegacyHintsMigrator.compactLegacyHints(LegacyHintsMigrator.java:108) > ~[main/:na] > at > org.apache.cassandra.hints.LegacyHintsMigrator.migrate(LegacyHintsMigrator.java:92) > ~[main/:na] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:321) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:710) > [main/:na] > Caused by: java.util.concurrent.ExecutionException: > java.lang.NegativeArraySizeException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > ~[na:1.8.0_51] > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > ~[na:1.8.0_51] > at > org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:115) > ~[main/:na] > ... 5 common frames omitted > Caused by: java.lang.NegativeArraySizeException: null > at > org.apache.cassandra.db.RowIndexEntry$LegacyShallowIndexedEntry.deserialize(RowIndexEntry.java:519) > ~[main/:na] > at > org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:321) > ~[main/:na] > at > org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:310) > ~[main/:na] > at > org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265) > ~[main/:na] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[main/:na] > at > org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:245) > ~[main/:na] > at > org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) > ~[main/:na] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) > ~[main/:na] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) > ~[main/:na] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[main/:na] > at > org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150) > ~[main/:na] > at > org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182) > ~[main/:na] > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82) > ~[main/:na] > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionManager$10.runMayThrow(CompactionManager.java:805) > ~[main/:na] > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > ~[main/:na] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >
[jira] [Commented] (CASSANDRA-11763) Failure to read non-shallow pre-3.0 sstable index entries
[ https://issues.apache.org/jira/browse/CASSANDRA-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310334#comment-15310334 ] T Jake Luciani commented on CASSANDRA-11763: looks like CASSANDRA-11393 > Failure to read non-shallow pre-3.0 sstable index entries > - > > Key: CASSANDRA-11763 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11763 > Project: Cassandra > Issue Type: Bug >Reporter: Philip Thompson >Assignee: Robert Stupp > Labels: dtest > Fix For: 3.6 > > Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log, > node3.log, node3_debug.log > > > example failure in: > http://cassci.datastax.com/view/Parameterized/job/upgrade_tests-all-custom_branch_runs/12/testReport/upgrade_tests.upgrade_through_versions_test/ProtoV3Upgrade_AllVersions_RandomPartitioner_Skips_3_0_x_EndsAt_Trunk_HEAD/rolling_upgrade_test_2/ > Logs are attached, relevant stack trace is here. > {code} > ERROR [main] 2016-05-11 16:26:06,555 CassandraDaemon.java:727 - Exception > encountered during startup > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.NegativeArraySizeException > at > org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:119) > ~[main/:na] > at > org.apache.cassandra.hints.LegacyHintsMigrator.compactLegacyHints(LegacyHintsMigrator.java:108) > ~[main/:na] > at > org.apache.cassandra.hints.LegacyHintsMigrator.migrate(LegacyHintsMigrator.java:92) > ~[main/:na] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:321) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:581) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:710) > [main/:na] > Caused by: java.util.concurrent.ExecutionException: > java.lang.NegativeArraySizeException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > ~[na:1.8.0_51] > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > ~[na:1.8.0_51] > at > org.apache.cassandra.hints.LegacyHintsMigrator.forceCompaction(LegacyHintsMigrator.java:115) > ~[main/:na] > ... 5 common frames omitted > Caused by: java.lang.NegativeArraySizeException: null > at > org.apache.cassandra.db.RowIndexEntry$LegacyShallowIndexedEntry.deserialize(RowIndexEntry.java:519) > ~[main/:na] > at > org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:321) > ~[main/:na] > at > org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:310) > ~[main/:na] > at > org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator.computeNext(BigTableScanner.java:265) > ~[main/:na] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[main/:na] > at > org.apache.cassandra.io.sstable.format.big.BigTableScanner.hasNext(BigTableScanner.java:245) > ~[main/:na] > at > org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) > ~[main/:na] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) > ~[main/:na] > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) > ~[main/:na] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[main/:na] > at > org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150) > ~[main/:na] > at > org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182) > ~[main/:na] > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82) > ~[main/:na] > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60) > ~[main/:na] > at > org.apache.cassandra.db.compaction.CompactionManager$10.runMayThrow(CompactionManager.java:805) > ~[main/:na] > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > ~[main/:na] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_51] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[na:1.8.0_51] >
[jira] [Commented] (CASSANDRA-11763) Failure to read non-shallow pre-3.0 sstable index entries
[ https://issues.apache.org/jira/browse/CASSANDRA-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310327#comment-15310327 ] Philip Thompson commented on CASSANDRA-11763: - [~tjake], [~snazy], I ran the tests and found an error in rolling_upgrade_test that I have never seen before and have not found a ticket for. Should I open a new bug? {code} Unexpected error in node2 log, error: ERROR [SharedPool-Worker-4] 2016-05-31 20:17:42,597 Message.java:611 - Unexpected exception during request; channel = [id: 0x152357bc, L:/127.0.0.2:9042 - R:/127.0.0.1:44896] java.lang.AssertionError: null at org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1267) ~[main/:na] at org.apache.cassandra.db.ReadCommand$LegacyReadCommandSerializer.serializedSize(ReadCommand.java:1214) ~[main/:na] at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:161) ~[main/:na] at org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72) ~[main/:na] at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609) ~[main/:na] at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758) ~[main/:na] at org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) ~[main/:na] at org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684) ~[main/:na] at org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110) ~[main/:na] at org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85) ~[main/:na] at org.apache.cassandra.service.AbstractReadExecutor$SpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:264) ~[main/:na] at org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1702) ~[main/:na] at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1657) ~[main/:na] at org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1604) ~[main/:na] at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1523) ~[main/:na] at org.apache.cassandra.db.SinglePartitionReadCommand.execute(SinglePartitionReadCommand.java:302) ~[main/:na] at org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:67) ~[main/:na] at org.apache.cassandra.service.pager.SinglePartitionPager.fetchPage(SinglePartitionPager.java:34) ~[main/:na] at org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:316) ~[main/:na] at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:352) ~[main/:na] at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:228) ~[main/:na] at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:78) ~[main/:na] at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:208) ~[main/:na] at org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:486) ~[main/:na] at org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:463) ~[main/:na] at org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130) ~[main/:na] at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507) [main/:na] at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401) [main/:na] at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-all-4.0.36.Final.jar:4.0.36.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292) [netty-all-4.0.36.Final.jar:4.0.36.Final] at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:32) [netty-all-4.0.36.Final.jar:4.0.36.Final] at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:283) [netty-all-4.0.36.Final.jar:4.0.36.Final] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51] at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) [main/:na] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [main/:na] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] {code} > Failure to read non-shallow pre-3.0 sstable index entries >
[jira] [Updated] (CASSANDRA-11152) SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file that you are sourcing
[ https://issues.apache.org/jira/browse/CASSANDRA-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-11152: -- Fix Version/s: (was: 3.8) > SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file > that you are sourcing > - > > Key: CASSANDRA-11152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11152 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: CQLSH 3.2.1 >Reporter: Francesco Animali >Assignee: Stefania > Labels: lhf > Fix For: 2.1.15, 2.2.7, 3.7, 3.0.7 > > > a difference in behaviour between SOURCE command in CQLSH 3.1 and 3.2. > In CQLSH 3.1 SOURCE will NOT require "use keyspace" in the cql file that you > execute: the "keyspace" directive in the qlshrc file will work and the cql > file will be executed. > In CQLSH 3.2.1, SOURCE command requires that "use keyspace" is in the cql > file that you are sourcing, otherwise it throws this error: > "No keyspace has been specified. USE a keyspace, or explicitly specify > keyspace.tablename". > The "keyspace" directive in cqlshrc is overridden by source command. > steps to reproduce: > create a file called select.cql in your home directory: > {noformat} > echo "CONSISTENCY ONE;" > select.cql > echo "select * from tab;" >> select.cql > {noformat} > in cqlsh: > {noformat} > create KEYSPACE kspace WITH replication = {'class': 'SimpleStrategy', > 'replication_factor': 1}; > create TABLE tab ( id int primary key); > insert into tab (id) VALUES ( 1); > {noformat} > Add this to cqlsgrc: > {noformat} > [authentication] > keyspace = kspace > {noformat} > Then exit cqlsh and rerun cqlsh using the cqlshrc just modified. > Note that you are in keyspace "kspace". > execute: > {noformat} > source 'select.cql' > {noformat} > this will have different behaviour in CQLSH 3.2 and 3.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7461) operator functionality in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310321#comment-15310321 ] Benjamin Lerer commented on CASSANDRA-7461: --- There are 3 categories of operators: * Arithmetic operators: {{+}}, {{-}}, {{*}}, {{/}} and {{%}} * Logical operators: {{|}}, {{&}}, {{!}} and {{XOR}} * String concatenation: {{+}} I think it is probably best to use this ticket as an umbrella ticket and to split the ticket into 3 sub task. > operator functionality in CQL > - > > Key: CASSANDRA-7461 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7461 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: Robert Stupp >Assignee: Benjamin Lerer > Labels: cql > > Intention: Allow operators in CQL > Operators could be decimal arithmetics {{+ - * /}} or boolen arithmetics {{| > & !}} or string 'arithmetics' {{+}} > {{SELECT tab.label + ' = ' + tab.value FROM foo.tab}} > {{SELECT * FROM tab WHERE tab.label + ' = ' + tab.value = 'foo = bar'}} > as well as > {{CREATE INDEX idx ON tab ( tab.tabel + '=' + tab.value )}} > or > {{CREATE INDEX idx ON tab (label) WHERE contains(tab.tabel, > 'very-important-key')}} > Operators could be mapped to UDFs like this: > {{+}} mapped to UDF {{cstarstd::oper_plus(...)}} > {{-}} mapped to UDF {{cstarstd::oper_minus(...)}} > or handled directly via {{Cql.g}} in 'special' code -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11152) SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file that you are sourcing
[ https://issues.apache.org/jira/browse/CASSANDRA-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310314#comment-15310314 ] Stefania commented on CASSANDRA-11152: -- Thanks, committed to 2.1 as 4008e9bd3fd8b6fca2f3c9f935b9f5dfa97116ad and merged upwards. > SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file > that you are sourcing > - > > Key: CASSANDRA-11152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11152 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: CQLSH 3.2.1 >Reporter: Francesco Animali >Assignee: Stefania > Labels: lhf > Fix For: 2.1.15, 2.2.7, 3.7, 3.0.7, 3.8 > > > a difference in behaviour between SOURCE command in CQLSH 3.1 and 3.2. > In CQLSH 3.1 SOURCE will NOT require "use keyspace" in the cql file that you > execute: the "keyspace" directive in the qlshrc file will work and the cql > file will be executed. > In CQLSH 3.2.1, SOURCE command requires that "use keyspace" is in the cql > file that you are sourcing, otherwise it throws this error: > "No keyspace has been specified. USE a keyspace, or explicitly specify > keyspace.tablename". > The "keyspace" directive in cqlshrc is overridden by source command. > steps to reproduce: > create a file called select.cql in your home directory: > {noformat} > echo "CONSISTENCY ONE;" > select.cql > echo "select * from tab;" >> select.cql > {noformat} > in cqlsh: > {noformat} > create KEYSPACE kspace WITH replication = {'class': 'SimpleStrategy', > 'replication_factor': 1}; > create TABLE tab ( id int primary key); > insert into tab (id) VALUES ( 1); > {noformat} > Add this to cqlsgrc: > {noformat} > [authentication] > keyspace = kspace > {noformat} > Then exit cqlsh and rerun cqlsh using the cqlshrc just modified. > Note that you are in keyspace "kspace". > execute: > {noformat} > source 'select.cql' > {noformat} > this will have different behaviour in CQLSH 3.2 and 3.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11152) SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file that you are sourcing
[ https://issues.apache.org/jira/browse/CASSANDRA-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-11152: - Resolution: Fixed Fix Version/s: (was: 3.0.x) (was: 2.2.x) (was: 2.1.x) (was: 3.x) 3.8 3.0.7 3.7 2.2.7 2.1.15 Status: Resolved (was: Ready to Commit) > SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file > that you are sourcing > - > > Key: CASSANDRA-11152 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11152 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: CQLSH 3.2.1 >Reporter: Francesco Animali >Assignee: Stefania > Labels: lhf > Fix For: 2.1.15, 2.2.7, 3.7, 3.0.7, 3.8 > > > a difference in behaviour between SOURCE command in CQLSH 3.1 and 3.2. > In CQLSH 3.1 SOURCE will NOT require "use keyspace" in the cql file that you > execute: the "keyspace" directive in the qlshrc file will work and the cql > file will be executed. > In CQLSH 3.2.1, SOURCE command requires that "use keyspace" is in the cql > file that you are sourcing, otherwise it throws this error: > "No keyspace has been specified. USE a keyspace, or explicitly specify > keyspace.tablename". > The "keyspace" directive in cqlshrc is overridden by source command. > steps to reproduce: > create a file called select.cql in your home directory: > {noformat} > echo "CONSISTENCY ONE;" > select.cql > echo "select * from tab;" >> select.cql > {noformat} > in cqlsh: > {noformat} > create KEYSPACE kspace WITH replication = {'class': 'SimpleStrategy', > 'replication_factor': 1}; > create TABLE tab ( id int primary key); > insert into tab (id) VALUES ( 1); > {noformat} > Add this to cqlsgrc: > {noformat} > [authentication] > keyspace = kspace > {noformat} > Then exit cqlsh and rerun cqlsh using the cqlshrc just modified. > Note that you are in keyspace "kspace". > execute: > {noformat} > source 'select.cql' > {noformat} > this will have different behaviour in CQLSH 3.2 and 3.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11055) C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no attribute 'replace'
[ https://issues.apache.org/jira/browse/CASSANDRA-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310317#comment-15310317 ] Aleksey Yeschenko commented on CASSANDRA-11055: --- We'll need to properly fix it before we remove Thrift in 4.0, but that can go into another ticket, sure. > C*2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns 'NoneType' object has no > attribute 'replace' > --- > > Key: CASSANDRA-11055 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11055 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Simon Ashley >Assignee: Stefania > Labels: cqlsh > Fix For: 2.1.x > > Attachments: 11055-driver-2.7.2.patch, data.tar.gz > > > C* 2.1 cqlsh DESCRIBE KEYSPACE ( or TABLE ) returns: > {code} > 'NoneType' object has no attribute 'replace' > {code} > for thrift CF's originally created in C* 1.2. > Repro: > 1. Create cf in cassandra-cli on C* 1.2.x (1.2.9 was used here) > {code} > [default@ks1] CREATE COLUMN FAMILY t1 > ... WITH column_type='Standard' > ... AND > comparator='CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)' > ... AND default_validation_class='UTF8Type' > ... AND key_validation_class='UTF8Type' > ... AND read_repair_chance=0.1 > ... AND dclocal_read_repair_chance=0.0 > ... AND gc_grace=864000 > ... AND min_compaction_threshold=4 > ... AND max_compaction_threshold=32 > ... AND replicate_on_write=true > ... AND compaction_strategy='LeveledCompactionStrategy' AND > compaction_strategy_options={sstable_size_in_mb: 32} > ... AND caching='KEYS_ONLY' > ... AND compression_options={sstable_compression:SnappyCompressor, > chunk_length_kb:64}; > qlsh> describe keyspace ks1; > CREATE KEYSPACE ks1 WITH replication = { > 'class': 'NetworkTopologyStrategy', > 'datacenter1': '1' > }; > USE ks1; > CREATE TABLE t1 ( > key text, > column1 text, > column2 text, > value text, > PRIMARY KEY (key, column1, column2) > ) WITH COMPACT STORAGE AND > bloom_filter_fp_chance=0.10 AND > caching='KEYS_ONLY' AND > comment='' AND > dclocal_read_repair_chance=0.00 AND > gc_grace_seconds=864000 AND > read_repair_chance=0.10 AND > replicate_on_write='true' AND > populate_io_cache_on_flush='false' AND > compaction={'sstable_size_in_mb': '32', 'class': > 'LeveledCompactionStrategy'} AND > compression={'chunk_length_kb': '64', 'sstable_compression': > 'SnappyCompressor'}; > cqlsh> select keyspace_name, columnfamily_name,column_aliases,key_aliases > from system.schema_columnfamilies where keyspace_name= 'ks1'; > keyspace_name | columnfamily_name | column_aliases | key_aliases > ---+---++- >ks1 |t1 | [] | [] > 2/ Upgrade -> C* 2.0.9 -> nodetool upgradesstables -a > At this stage , DESCRIBE in cqlsh is working > 3/ Upgrade -> C* 2.1.12 -> nodetool upgradesstables -a > DESCRIBE now fails: > cqlsh> describe table ks1.t1; > 'NoneType' object has no attribute 'replace' > cqlsh> describe keyspace ks1; > 'NoneType' object has no attribute 'replace' > {code} > You can workaround by manually updating {{system.schema_columnfamilies}} > {code} > UPDATE system.schema_columnfamilies SET column_aliases > ='["column1","column2"]' WHERE keyspace_name = 'ks1' AND columnfamily_name = > 't1'; > {code} > Once you exit and restart cqlsh, {{DESCRIBE}} is not working as per C* 1.2 > {code} > cqlsh> describe keyspace ks1; > CREATE KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', > 'datacenter1': '1'} AND durable_writes = true; > CREATE TABLE ks1.t1 ( > key text, > column1 text, > column2 text, > value text, > PRIMARY KEY (key, column1, column2) > ) WITH COMPACT STORAGE > AND CLUSTERING ORDER BY (column1 ASC, column2 ASC) > AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' > AND comment = '' > AND compaction = {'sstable_size_in_mb': '32', 'class': > 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'} > AND compression = {'chunk_length_kb': '64', 'sstable_compression': > 'org.apache.cassandra.io.compress.SnappyCompressor'} > AND dclocal_read_repair_chance = 0.0 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.1 > AND speculative_retry = '99.0PERCENTILE'; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[15/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f99f8a8c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f99f8a8c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f99f8a8c Branch: refs/heads/cassandra-3.7 Commit: f99f8a8c823c01cc5ef781fdf36ecc4725f4d947 Parents: 4aaf2b3 43e9d29 Author: Stefania AlborghettiAuthored: Wed Jun 1 15:32:48 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:32:48 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f99f8a8c/CHANGES.txt -- diff --cc CHANGES.txt index 4cdc031,ffd5b31..07845a6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -16,39 -3,9 +16,40 @@@ Merged from 2.2 * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) * Possible memory leak in NIODataInputStream (CASSANDRA-11867) - * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) - * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395) +Merged from 2.1: ++ * cqlsh: apply current keyspace to source command (CASSANDRA-11152) + * Backport CASSANDRA-11578 (CASSANDRA-11750) + * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) + * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) + * Do not consider local node a valid source during replace (CASSANDRA-11848) + * Add message dropped tasks to nodetool netstats (CASSANDRA-11855) + * Avoid holding SSTableReaders for duration of incremental repair (CASSANDRA-11739) + + +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f99f8a8c/bin/cqlsh.py --
[11/19] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43e9d293 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43e9d293 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43e9d293 Branch: refs/heads/cassandra-3.0 Commit: 43e9d2936e1d352b98433ecb1d3c2608e641870e Parents: 39b86e3 4008e9b Author: Stefania AlborghettiAuthored: Wed Jun 1 15:30:42 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:30:57 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e9d293/CHANGES.txt -- diff --cc CHANGES.txt index 7c66125,a437322..ffd5b31 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,5 +1,27 @@@ -2.1.15 +2.2.7 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840)
[08/19] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e9d293/bin/cqlsh.py -- diff --cc bin/cqlsh.py index 85605ae,000..dd0446a mode 100644,00..100644 --- a/bin/cqlsh.py +++ b/bin/cqlsh.py @@@ -1,2579 -1,0 +1,2580 @@@ +#!/bin/sh +# -*- mode: Python -*- + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""":" +# bash code here; finds a suitable python interpreter and execs this file. +# prefer unqualified "python" if suitable: +python -c 'import sys; sys.exit(not (0x020700b0 < sys.hexversion < 0x0300))' 2>/dev/null \ +&& exec python "$0" "$@" +for pyver in 2.7; do +which python$pyver > /dev/null 2>&1 && exec python$pyver "$0" "$@" +done +echo "No appropriate python interpreter found." >&2 +exit 1 +":""" + +from __future__ import with_statement + +import cmd +import codecs +import ConfigParser +import csv +import getpass +import optparse +import os +import platform +import sys +import traceback +import warnings +import webbrowser +from StringIO import StringIO +from contextlib import contextmanager +from glob import glob +from uuid import UUID + +if sys.version_info[0] != 2 or sys.version_info[1] != 7: +sys.exit("\nCQL Shell supports only Python 2.7\n") + +UTF8 = 'utf-8' +CP65001 = 'cp65001' # Win utf-8 variant + +description = "CQL Shell for Apache Cassandra" +version = "5.0.1" + +readline = None +try: +# check if tty first, cause readline doesn't check, and only cares +# about $TERM. we don't want the funky escape code stuff to be +# output if not a tty. +if sys.stdin.isatty(): +import readline +except ImportError: +pass + +CQL_LIB_PREFIX = 'cassandra-driver-internal-only-' + +CASSANDRA_PATH = os.path.join(os.path.dirname(os.path.realpath(__file__)), '..') +CASSANDRA_CQL_HTML_FALLBACK = 'https://cassandra.apache.org/doc/cql3/CQL-2.2.html' + +if os.path.exists(CASSANDRA_PATH + '/doc/cql3/CQL.html'): +# default location of local CQL.html +CASSANDRA_CQL_HTML = 'file://' + CASSANDRA_PATH + '/doc/cql3/CQL.html' +elif os.path.exists('/usr/share/doc/cassandra/CQL.html'): +# fallback to package file +CASSANDRA_CQL_HTML = 'file:///usr/share/doc/cassandra/CQL.html' +else: +# fallback to online version +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK + +# On Linux, the Python webbrowser module uses the 'xdg-open' executable +# to open a file/URL. But that only works, if the current session has been +# opened from _within_ a desktop environment. I.e. 'xdg-open' will fail, +# if the session's been opened via ssh to a remote box. +# +# Use 'python' to get some information about the detected browsers. +# >>> import webbrowser +# >>> webbrowser._tryorder +# >>> webbrowser._browser +# +if len(webbrowser._tryorder) == 0: +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK +elif webbrowser._tryorder[0] == 'xdg-open' and os.environ.get('XDG_DATA_DIRS', '') == '': +# only on Linux (some OS with xdg-open) +webbrowser._tryorder.remove('xdg-open') +webbrowser._tryorder.append('xdg-open') + +# use bundled libs for python-cql and thrift, if available. if there +# is a ../lib dir, use bundled libs there preferentially. +ZIPLIB_DIRS = [os.path.join(CASSANDRA_PATH, 'lib')] +myplatform = platform.system() +is_win = myplatform == 'Windows' + +# Workaround for supporting CP65001 encoding on python < 3.3 (https://bugs.python.org/issue13216) +if is_win and sys.version_info < (3, 3): +codecs.register(lambda name: codecs.lookup(UTF8) if name == CP65001 else None) + +if myplatform == 'Linux': +ZIPLIB_DIRS.append('/usr/share/cassandra/lib') + +if os.environ.get('CQLSH_NO_BUNDLED', ''): +ZIPLIB_DIRS = () + + +def find_zip(libprefix): +for ziplibdir in ZIPLIB_DIRS: +zips = glob(os.path.join(ziplibdir, libprefix + '*.zip')) +if zips: +return max(zips) # probably the highest version, if multiple + +cql_zip = find_zip(CQL_LIB_PREFIX) +if cql_zip: +ver = os.path.splitext(os.path.basename(cql_zip))[0][len(CQL_LIB_PREFIX):] +sys.path.insert(0, os.path.join(cql_zip,
[14/19] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f99f8a8c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f99f8a8c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f99f8a8c Branch: refs/heads/trunk Commit: f99f8a8c823c01cc5ef781fdf36ecc4725f4d947 Parents: 4aaf2b3 43e9d29 Author: Stefania AlborghettiAuthored: Wed Jun 1 15:32:48 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:32:48 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f99f8a8c/CHANGES.txt -- diff --cc CHANGES.txt index 4cdc031,ffd5b31..07845a6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -16,39 -3,9 +16,40 @@@ Merged from 2.2 * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) * Possible memory leak in NIODataInputStream (CASSANDRA-11867) - * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) - * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395) +Merged from 2.1: ++ * cqlsh: apply current keyspace to source command (CASSANDRA-11152) + * Backport CASSANDRA-11578 (CASSANDRA-11750) + * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) + * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) + * Do not consider local node a valid source during replace (CASSANDRA-11848) + * Add message dropped tasks to nodetool netstats (CASSANDRA-11855) + * Avoid holding SSTableReaders for duration of incremental repair (CASSANDRA-11739) + + +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f99f8a8c/bin/cqlsh.py --
[06/19] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e9d293/bin/cqlsh.py -- diff --cc bin/cqlsh.py index 85605ae,000..dd0446a mode 100644,00..100644 --- a/bin/cqlsh.py +++ b/bin/cqlsh.py @@@ -1,2579 -1,0 +1,2580 @@@ +#!/bin/sh +# -*- mode: Python -*- + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""":" +# bash code here; finds a suitable python interpreter and execs this file. +# prefer unqualified "python" if suitable: +python -c 'import sys; sys.exit(not (0x020700b0 < sys.hexversion < 0x0300))' 2>/dev/null \ +&& exec python "$0" "$@" +for pyver in 2.7; do +which python$pyver > /dev/null 2>&1 && exec python$pyver "$0" "$@" +done +echo "No appropriate python interpreter found." >&2 +exit 1 +":""" + +from __future__ import with_statement + +import cmd +import codecs +import ConfigParser +import csv +import getpass +import optparse +import os +import platform +import sys +import traceback +import warnings +import webbrowser +from StringIO import StringIO +from contextlib import contextmanager +from glob import glob +from uuid import UUID + +if sys.version_info[0] != 2 or sys.version_info[1] != 7: +sys.exit("\nCQL Shell supports only Python 2.7\n") + +UTF8 = 'utf-8' +CP65001 = 'cp65001' # Win utf-8 variant + +description = "CQL Shell for Apache Cassandra" +version = "5.0.1" + +readline = None +try: +# check if tty first, cause readline doesn't check, and only cares +# about $TERM. we don't want the funky escape code stuff to be +# output if not a tty. +if sys.stdin.isatty(): +import readline +except ImportError: +pass + +CQL_LIB_PREFIX = 'cassandra-driver-internal-only-' + +CASSANDRA_PATH = os.path.join(os.path.dirname(os.path.realpath(__file__)), '..') +CASSANDRA_CQL_HTML_FALLBACK = 'https://cassandra.apache.org/doc/cql3/CQL-2.2.html' + +if os.path.exists(CASSANDRA_PATH + '/doc/cql3/CQL.html'): +# default location of local CQL.html +CASSANDRA_CQL_HTML = 'file://' + CASSANDRA_PATH + '/doc/cql3/CQL.html' +elif os.path.exists('/usr/share/doc/cassandra/CQL.html'): +# fallback to package file +CASSANDRA_CQL_HTML = 'file:///usr/share/doc/cassandra/CQL.html' +else: +# fallback to online version +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK + +# On Linux, the Python webbrowser module uses the 'xdg-open' executable +# to open a file/URL. But that only works, if the current session has been +# opened from _within_ a desktop environment. I.e. 'xdg-open' will fail, +# if the session's been opened via ssh to a remote box. +# +# Use 'python' to get some information about the detected browsers. +# >>> import webbrowser +# >>> webbrowser._tryorder +# >>> webbrowser._browser +# +if len(webbrowser._tryorder) == 0: +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK +elif webbrowser._tryorder[0] == 'xdg-open' and os.environ.get('XDG_DATA_DIRS', '') == '': +# only on Linux (some OS with xdg-open) +webbrowser._tryorder.remove('xdg-open') +webbrowser._tryorder.append('xdg-open') + +# use bundled libs for python-cql and thrift, if available. if there +# is a ../lib dir, use bundled libs there preferentially. +ZIPLIB_DIRS = [os.path.join(CASSANDRA_PATH, 'lib')] +myplatform = platform.system() +is_win = myplatform == 'Windows' + +# Workaround for supporting CP65001 encoding on python < 3.3 (https://bugs.python.org/issue13216) +if is_win and sys.version_info < (3, 3): +codecs.register(lambda name: codecs.lookup(UTF8) if name == CP65001 else None) + +if myplatform == 'Linux': +ZIPLIB_DIRS.append('/usr/share/cassandra/lib') + +if os.environ.get('CQLSH_NO_BUNDLED', ''): +ZIPLIB_DIRS = () + + +def find_zip(libprefix): +for ziplibdir in ZIPLIB_DIRS: +zips = glob(os.path.join(ziplibdir, libprefix + '*.zip')) +if zips: +return max(zips) # probably the highest version, if multiple + +cql_zip = find_zip(CQL_LIB_PREFIX) +if cql_zip: +ver = os.path.splitext(os.path.basename(cql_zip))[0][len(CQL_LIB_PREFIX):] +sys.path.insert(0, os.path.join(cql_zip,
[02/19] cassandra git commit: cqlsh: apply current keyspace to source command
cqlsh: apply current keyspace to source command patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-11152 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4008e9bd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4008e9bd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4008e9bd Branch: refs/heads/cassandra-2.2 Commit: 4008e9bd3fd8b6fca2f3c9f935b9f5dfa97116ad Parents: b851792 Author: Stefania AlborghettiAuthored: Tue May 31 16:52:26 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:25:40 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ad9d00c..a437322 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.15 + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/bin/cqlsh -- diff --git a/bin/cqlsh b/bin/cqlsh index 374e588..6317ec9 100755 --- a/bin/cqlsh +++ b/bin/cqlsh @@ -1676,6 +1676,7 @@ class Shell(cmd.Cmd): subshell = Shell(self.hostname, self.port, color=self.color, encoding=self.encoding, stdin=f, tty=False, use_conn=self.conn, cqlver=self.cql_version, + keyspace=self.current_keyspace, display_time_format=self.display_time_format, display_float_precision=self.display_float_precision, max_trace_wait=self.max_trace_wait)
[12/19] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e9d293/bin/cqlsh.py -- diff --cc bin/cqlsh.py index 85605ae,000..dd0446a mode 100644,00..100644 --- a/bin/cqlsh.py +++ b/bin/cqlsh.py @@@ -1,2579 -1,0 +1,2580 @@@ +#!/bin/sh +# -*- mode: Python -*- + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""":" +# bash code here; finds a suitable python interpreter and execs this file. +# prefer unqualified "python" if suitable: +python -c 'import sys; sys.exit(not (0x020700b0 < sys.hexversion < 0x0300))' 2>/dev/null \ +&& exec python "$0" "$@" +for pyver in 2.7; do +which python$pyver > /dev/null 2>&1 && exec python$pyver "$0" "$@" +done +echo "No appropriate python interpreter found." >&2 +exit 1 +":""" + +from __future__ import with_statement + +import cmd +import codecs +import ConfigParser +import csv +import getpass +import optparse +import os +import platform +import sys +import traceback +import warnings +import webbrowser +from StringIO import StringIO +from contextlib import contextmanager +from glob import glob +from uuid import UUID + +if sys.version_info[0] != 2 or sys.version_info[1] != 7: +sys.exit("\nCQL Shell supports only Python 2.7\n") + +UTF8 = 'utf-8' +CP65001 = 'cp65001' # Win utf-8 variant + +description = "CQL Shell for Apache Cassandra" +version = "5.0.1" + +readline = None +try: +# check if tty first, cause readline doesn't check, and only cares +# about $TERM. we don't want the funky escape code stuff to be +# output if not a tty. +if sys.stdin.isatty(): +import readline +except ImportError: +pass + +CQL_LIB_PREFIX = 'cassandra-driver-internal-only-' + +CASSANDRA_PATH = os.path.join(os.path.dirname(os.path.realpath(__file__)), '..') +CASSANDRA_CQL_HTML_FALLBACK = 'https://cassandra.apache.org/doc/cql3/CQL-2.2.html' + +if os.path.exists(CASSANDRA_PATH + '/doc/cql3/CQL.html'): +# default location of local CQL.html +CASSANDRA_CQL_HTML = 'file://' + CASSANDRA_PATH + '/doc/cql3/CQL.html' +elif os.path.exists('/usr/share/doc/cassandra/CQL.html'): +# fallback to package file +CASSANDRA_CQL_HTML = 'file:///usr/share/doc/cassandra/CQL.html' +else: +# fallback to online version +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK + +# On Linux, the Python webbrowser module uses the 'xdg-open' executable +# to open a file/URL. But that only works, if the current session has been +# opened from _within_ a desktop environment. I.e. 'xdg-open' will fail, +# if the session's been opened via ssh to a remote box. +# +# Use 'python' to get some information about the detected browsers. +# >>> import webbrowser +# >>> webbrowser._tryorder +# >>> webbrowser._browser +# +if len(webbrowser._tryorder) == 0: +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK +elif webbrowser._tryorder[0] == 'xdg-open' and os.environ.get('XDG_DATA_DIRS', '') == '': +# only on Linux (some OS with xdg-open) +webbrowser._tryorder.remove('xdg-open') +webbrowser._tryorder.append('xdg-open') + +# use bundled libs for python-cql and thrift, if available. if there +# is a ../lib dir, use bundled libs there preferentially. +ZIPLIB_DIRS = [os.path.join(CASSANDRA_PATH, 'lib')] +myplatform = platform.system() +is_win = myplatform == 'Windows' + +# Workaround for supporting CP65001 encoding on python < 3.3 (https://bugs.python.org/issue13216) +if is_win and sys.version_info < (3, 3): +codecs.register(lambda name: codecs.lookup(UTF8) if name == CP65001 else None) + +if myplatform == 'Linux': +ZIPLIB_DIRS.append('/usr/share/cassandra/lib') + +if os.environ.get('CQLSH_NO_BUNDLED', ''): +ZIPLIB_DIRS = () + + +def find_zip(libprefix): +for ziplibdir in ZIPLIB_DIRS: +zips = glob(os.path.join(ziplibdir, libprefix + '*.zip')) +if zips: +return max(zips) # probably the highest version, if multiple + +cql_zip = find_zip(CQL_LIB_PREFIX) +if cql_zip: +ver = os.path.splitext(os.path.basename(cql_zip))[0][len(CQL_LIB_PREFIX):] +sys.path.insert(0, os.path.join(cql_zip,
[03/19] cassandra git commit: cqlsh: apply current keyspace to source command
cqlsh: apply current keyspace to source command patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-11152 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4008e9bd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4008e9bd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4008e9bd Branch: refs/heads/trunk Commit: 4008e9bd3fd8b6fca2f3c9f935b9f5dfa97116ad Parents: b851792 Author: Stefania AlborghettiAuthored: Tue May 31 16:52:26 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:25:40 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ad9d00c..a437322 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.15 + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/bin/cqlsh -- diff --git a/bin/cqlsh b/bin/cqlsh index 374e588..6317ec9 100755 --- a/bin/cqlsh +++ b/bin/cqlsh @@ -1676,6 +1676,7 @@ class Shell(cmd.Cmd): subshell = Shell(self.hostname, self.port, color=self.color, encoding=self.encoding, stdin=f, tty=False, use_conn=self.conn, cqlver=self.cql_version, + keyspace=self.current_keyspace, display_time_format=self.display_time_format, display_float_precision=self.display_float_precision, max_trace_wait=self.max_trace_wait)
[13/19] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43e9d293 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43e9d293 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43e9d293 Branch: refs/heads/cassandra-3.7 Commit: 43e9d2936e1d352b98433ecb1d3c2608e641870e Parents: 39b86e3 4008e9b Author: Stefania AlborghettiAuthored: Wed Jun 1 15:30:42 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:30:57 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e9d293/CHANGES.txt -- diff --cc CHANGES.txt index 7c66125,a437322..ffd5b31 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,5 +1,27 @@@ -2.1.15 +2.2.7 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840)
[19/19] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/274a8e4f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/274a8e4f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/274a8e4f Branch: refs/heads/trunk Commit: 274a8e4ffec79564edae17e43bbf4074170f6a5d Parents: f55d2d4 01847ac Author: Stefania AlborghettiAuthored: Wed Jun 1 15:34:04 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:34:04 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/274a8e4f/CHANGES.txt --
[05/19] cassandra git commit: cqlsh: apply current keyspace to source command
cqlsh: apply current keyspace to source command patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-11152 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4008e9bd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4008e9bd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4008e9bd Branch: refs/heads/cassandra-3.7 Commit: 4008e9bd3fd8b6fca2f3c9f935b9f5dfa97116ad Parents: b851792 Author: Stefania AlborghettiAuthored: Tue May 31 16:52:26 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:25:40 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ad9d00c..a437322 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.15 + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/bin/cqlsh -- diff --git a/bin/cqlsh b/bin/cqlsh index 374e588..6317ec9 100755 --- a/bin/cqlsh +++ b/bin/cqlsh @@ -1676,6 +1676,7 @@ class Shell(cmd.Cmd): subshell = Shell(self.hostname, self.port, color=self.color, encoding=self.encoding, stdin=f, tty=False, use_conn=self.conn, cqlver=self.cql_version, + keyspace=self.current_keyspace, display_time_format=self.display_time_format, display_float_precision=self.display_float_precision, max_trace_wait=self.max_trace_wait)
[17/19] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/01847ac5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/01847ac5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/01847ac5 Branch: refs/heads/cassandra-3.7 Commit: 01847ac51064f894d0c6125fa36f3958f7846ba8 Parents: f000ff0 f99f8a8 Author: Stefania AlborghettiAuthored: Wed Jun 1 15:33:45 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:33:45 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/01847ac5/CHANGES.txt -- diff --cc CHANGES.txt index 82b4502,07845a6..c6cca1a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -20,10 -17,10 +20,11 @@@ Merged from 2.2 * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) * Possible memory leak in NIODataInputStream (CASSANDRA-11867) * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) Merged from 2.1: + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) - * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) * Do not consider local node a valid source during replace (CASSANDRA-11848) http://git-wip-us.apache.org/repos/asf/cassandra/blob/01847ac5/bin/cqlsh.py --
[10/19] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e9d293/bin/cqlsh.py -- diff --cc bin/cqlsh.py index 85605ae,000..dd0446a mode 100644,00..100644 --- a/bin/cqlsh.py +++ b/bin/cqlsh.py @@@ -1,2579 -1,0 +1,2580 @@@ +#!/bin/sh +# -*- mode: Python -*- + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""":" +# bash code here; finds a suitable python interpreter and execs this file. +# prefer unqualified "python" if suitable: +python -c 'import sys; sys.exit(not (0x020700b0 < sys.hexversion < 0x0300))' 2>/dev/null \ +&& exec python "$0" "$@" +for pyver in 2.7; do +which python$pyver > /dev/null 2>&1 && exec python$pyver "$0" "$@" +done +echo "No appropriate python interpreter found." >&2 +exit 1 +":""" + +from __future__ import with_statement + +import cmd +import codecs +import ConfigParser +import csv +import getpass +import optparse +import os +import platform +import sys +import traceback +import warnings +import webbrowser +from StringIO import StringIO +from contextlib import contextmanager +from glob import glob +from uuid import UUID + +if sys.version_info[0] != 2 or sys.version_info[1] != 7: +sys.exit("\nCQL Shell supports only Python 2.7\n") + +UTF8 = 'utf-8' +CP65001 = 'cp65001' # Win utf-8 variant + +description = "CQL Shell for Apache Cassandra" +version = "5.0.1" + +readline = None +try: +# check if tty first, cause readline doesn't check, and only cares +# about $TERM. we don't want the funky escape code stuff to be +# output if not a tty. +if sys.stdin.isatty(): +import readline +except ImportError: +pass + +CQL_LIB_PREFIX = 'cassandra-driver-internal-only-' + +CASSANDRA_PATH = os.path.join(os.path.dirname(os.path.realpath(__file__)), '..') +CASSANDRA_CQL_HTML_FALLBACK = 'https://cassandra.apache.org/doc/cql3/CQL-2.2.html' + +if os.path.exists(CASSANDRA_PATH + '/doc/cql3/CQL.html'): +# default location of local CQL.html +CASSANDRA_CQL_HTML = 'file://' + CASSANDRA_PATH + '/doc/cql3/CQL.html' +elif os.path.exists('/usr/share/doc/cassandra/CQL.html'): +# fallback to package file +CASSANDRA_CQL_HTML = 'file:///usr/share/doc/cassandra/CQL.html' +else: +# fallback to online version +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK + +# On Linux, the Python webbrowser module uses the 'xdg-open' executable +# to open a file/URL. But that only works, if the current session has been +# opened from _within_ a desktop environment. I.e. 'xdg-open' will fail, +# if the session's been opened via ssh to a remote box. +# +# Use 'python' to get some information about the detected browsers. +# >>> import webbrowser +# >>> webbrowser._tryorder +# >>> webbrowser._browser +# +if len(webbrowser._tryorder) == 0: +CASSANDRA_CQL_HTML = CASSANDRA_CQL_HTML_FALLBACK +elif webbrowser._tryorder[0] == 'xdg-open' and os.environ.get('XDG_DATA_DIRS', '') == '': +# only on Linux (some OS with xdg-open) +webbrowser._tryorder.remove('xdg-open') +webbrowser._tryorder.append('xdg-open') + +# use bundled libs for python-cql and thrift, if available. if there +# is a ../lib dir, use bundled libs there preferentially. +ZIPLIB_DIRS = [os.path.join(CASSANDRA_PATH, 'lib')] +myplatform = platform.system() +is_win = myplatform == 'Windows' + +# Workaround for supporting CP65001 encoding on python < 3.3 (https://bugs.python.org/issue13216) +if is_win and sys.version_info < (3, 3): +codecs.register(lambda name: codecs.lookup(UTF8) if name == CP65001 else None) + +if myplatform == 'Linux': +ZIPLIB_DIRS.append('/usr/share/cassandra/lib') + +if os.environ.get('CQLSH_NO_BUNDLED', ''): +ZIPLIB_DIRS = () + + +def find_zip(libprefix): +for ziplibdir in ZIPLIB_DIRS: +zips = glob(os.path.join(ziplibdir, libprefix + '*.zip')) +if zips: +return max(zips) # probably the highest version, if multiple + +cql_zip = find_zip(CQL_LIB_PREFIX) +if cql_zip: +ver = os.path.splitext(os.path.basename(cql_zip))[0][len(CQL_LIB_PREFIX):] +sys.path.insert(0, os.path.join(cql_zip,
[09/19] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43e9d293 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43e9d293 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43e9d293 Branch: refs/heads/cassandra-2.2 Commit: 43e9d2936e1d352b98433ecb1d3c2608e641870e Parents: 39b86e3 4008e9b Author: Stefania AlborghettiAuthored: Wed Jun 1 15:30:42 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:30:57 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/43e9d293/CHANGES.txt -- diff --cc CHANGES.txt index 7c66125,a437322..ffd5b31 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,5 +1,27 @@@ -2.1.15 +2.2.7 + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840)
[04/19] cassandra git commit: cqlsh: apply current keyspace to source command
cqlsh: apply current keyspace to source command patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-11152 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4008e9bd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4008e9bd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4008e9bd Branch: refs/heads/cassandra-3.0 Commit: 4008e9bd3fd8b6fca2f3c9f935b9f5dfa97116ad Parents: b851792 Author: Stefania AlborghettiAuthored: Tue May 31 16:52:26 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:25:40 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ad9d00c..a437322 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.15 + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/bin/cqlsh -- diff --git a/bin/cqlsh b/bin/cqlsh index 374e588..6317ec9 100755 --- a/bin/cqlsh +++ b/bin/cqlsh @@ -1676,6 +1676,7 @@ class Shell(cmd.Cmd): subshell = Shell(self.hostname, self.port, color=self.color, encoding=self.encoding, stdin=f, tty=False, use_conn=self.conn, cqlver=self.cql_version, + keyspace=self.current_keyspace, display_time_format=self.display_time_format, display_float_precision=self.display_float_precision, max_trace_wait=self.max_trace_wait)
[18/19] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/01847ac5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/01847ac5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/01847ac5 Branch: refs/heads/trunk Commit: 01847ac51064f894d0c6125fa36f3958f7846ba8 Parents: f000ff0 f99f8a8 Author: Stefania AlborghettiAuthored: Wed Jun 1 15:33:45 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:33:45 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh.py | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/01847ac5/CHANGES.txt -- diff --cc CHANGES.txt index 82b4502,07845a6..c6cca1a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -20,10 -17,10 +20,11 @@@ Merged from 2.2 * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) * Possible memory leak in NIODataInputStream (CASSANDRA-11867) * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) * Prohibit Reversed Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) Merged from 2.1: + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) - * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) * Do not consider local node a valid source during replace (CASSANDRA-11848) http://git-wip-us.apache.org/repos/asf/cassandra/blob/01847ac5/bin/cqlsh.py --
[01/19] cassandra git commit: cqlsh: apply current keyspace to source command
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 b851792c4 -> 4008e9bd3 refs/heads/cassandra-2.2 39b86e379 -> 43e9d2936 refs/heads/cassandra-3.0 4aaf2b34f -> f99f8a8c8 refs/heads/cassandra-3.7 f000ff08f -> 01847ac51 refs/heads/trunk f55d2d494 -> 274a8e4ff cqlsh: apply current keyspace to source command patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-11152 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4008e9bd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4008e9bd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4008e9bd Branch: refs/heads/cassandra-2.1 Commit: 4008e9bd3fd8b6fca2f3c9f935b9f5dfa97116ad Parents: b851792 Author: Stefania AlborghettiAuthored: Tue May 31 16:52:26 2016 +0200 Committer: Stefania Alborghetti Committed: Wed Jun 1 15:25:40 2016 +0200 -- CHANGES.txt | 1 + bin/cqlsh | 1 + 2 files changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ad9d00c..a437322 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.15 + * cqlsh: apply current keyspace to source command (CASSANDRA-11152) * Backport CASSANDRA-11578 (CASSANDRA-11750) * Clear out parent repair session if repair coordinator dies (CASSANDRA-11824) * Set default streaming_socket_timeout_in_ms to 24 hours (CASSANDRA-11840) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4008e9bd/bin/cqlsh -- diff --git a/bin/cqlsh b/bin/cqlsh index 374e588..6317ec9 100755 --- a/bin/cqlsh +++ b/bin/cqlsh @@ -1676,6 +1676,7 @@ class Shell(cmd.Cmd): subshell = Shell(self.hostname, self.port, color=self.color, encoding=self.encoding, stdin=f, tty=False, use_conn=self.conn, cqlver=self.cql_version, + keyspace=self.current_keyspace, display_time_format=self.display_time_format, display_float_precision=self.display_float_precision, max_trace_wait=self.max_trace_wait)