[jira] [Commented] (CASSANDRA-10085) Allow BATCH with conditions to span multiple tables with same partition key
[ https://issues.apache.org/jira/browse/CASSANDRA-10085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733311#comment-14733311 ] Dennis Roppelt commented on CASSANDRA-10085: Hello Sylvain, Thank you for the quick and informative reply. I already anticipated that this would not be trivial to implement, and might go against some design principles created in C*. I also can see why this won't be implemented soon or at all. I will look into a different approach of modelling my data or see if another database is a better fit (obviously standard RDBMS will work for the model). It is a bit disappointing though, that this limitation was not mentioned in the documentation. Compared to other NoSQL-DBs, Cassandra has an outstandingly good documentation. How would I suggest adding a hint about the limitiation with batch + conditional statements across tables? Creating a new issue and link to this one? > Allow BATCH with conditions to span multiple tables with same partition key > --- > > Key: CASSANDRA-10085 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10085 > Project: Cassandra > Issue Type: Improvement >Reporter: Dennis Roppelt > Labels: batch, conditional-statement, partitioning > > h4. Use case: > I want to use batch to keep data between tables synchronized with help of > conditionals. In particular, I want to have a ONE-to-ONE relationship in my > data model. I dont want to have an unintended upsert while trying to maintain > a consistent one-to-one relationship (write requests from other clients that > try to create a different relationship with one of the keys I inteded to > use), which is why I would like the IF NOT EXISTS when inserting data into > multiple tables in a batch. > But when trying to insert data in a batch-block with an IF NOT EXISTS, I get > the following response (reproduceable example further below): > {code} > cqlsh:testkeyspace> > BEGIN BATCH > INSERT INTO key1_to_key2 (partKey, key1, key2) values (1,2,3) IF NOT EXISTS; > INSERT INTO key2_to_key1 (partKey, key1, key2) values (1,2,3) IF NOT EXISTS; > APPLY BATCH; > Bad Request: Batch with conditions cannot span multiple tables{code} > The tables have a primary key on (partKey, key1) and (partKey, key2). Which > means that partKey in both cases is their partition key. Which in the example > also is the same value. > h4. Why I want to use a BATCH-statement that way: > In traditional databases, to design a one-to-one relationship, I would create > one table with two keys, both with a unique constraint: > {code:sql} > CREATE TABLE `myOneToOneTable` ( > `ID` int NOT NULL, > `KEY_ONE` int NOT NULL, > `KEY_TWO` int NOT NULL, > // some other fields > PRIMARY KEY (`ID`), > UNIQUE KEY `KEY_ONE`, > UNIQUE KEY `KEY_TWO` > ); > {code} > (simplified example taken from an [one-to-one example for > mysql|http://www.mkyong.com/mysql/how-to-define-one-to-one-relationship-in-mysql/]) > As far as I know, unique constraints are not supported in Cassandra, so > another way around it would be to maintain the relationshipt between two > tables, each sharing the same partition key. For which I would like Cassandra > to make sure that when I insert data, either both are inserted at the same > time or none (to prevent other writing client to insert a different > one-to-one relationship than me). > [According to the documentation, this is what BATCH-Statements are used > for|http://docs.datastax.com/en/cql/3.3/cql/cql_using/useBatch.html] > {quote} > Instead, using batches to synchronize data to tables is a legitimate > operation. > {quote} > I expected it to be able to do it, but as seen above, I cannot enforce INSERT > IF NOT EXISTS over multiple tables. > I do understand the background as to *why* there is a limitation coded that > way, so that not too many nodes have to take part in processing of the batch. > However, if the operations use > # primary keys in their statements for each table > # all primary keys contain the same value, thus resulting in the same hash > for partioning > # tables belong to the same keyspace > the batch statement should be able to be limited to the same amount of nodes > as if the batch contained INSERT IF NOT EXISTs for only one table. > h4. Steps to reproduce desired behaviour: > I used the [cassandra ova on virtual > box|http://www.planetcassandra.org/install-cassandra-ova-on-virtualbox/], but > also a Windows Setup with 2.2.0 > {code} > Connected to Cluster on a Stick at localhost:9160. > > [cqlsh 4.1.1 | Cassandra 2.0.7 | CQL spec 3.1.1 | Thrift protocol 19.39.0] > > Use HELP for help. > > cqlsh> CREATE KEYSPACE conditionalBatch
[jira] [Commented] (CASSANDRA-10255) AttributeError: 'module' object has no attribute 'compress'
[ https://issues.apache.org/jira/browse/CASSANDRA-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733312#comment-14733312 ] navithejesh commented on CASSANDRA-10255: - Hi Tyler , -->I have checked the some other single node, which has output message for python -c 'import snappy; print dir(snappy); print snappy._file_' root@orange-tree:~# python -c 'import snappy; print dir(snappy);print snappy.__file__' ['CompressError', 'CompressedLengthError', 'InvalidCompressedInputError', 'UncompressError', '__doc__', '__file__', '__name__', '__package__', 'compress', 'decompress', 'isValidCompressed', 'uncompress'] /usr/lib/python2.7/dist-packages/snappy.so --> Here we see compress and uncompress modules are present. And I checked for the same in node B and it was found in "/usr/lib64/python2.7/site-packages/". I moved snappy.py from the "/usr/lib64/python2.7/site-packages/" to "/usr/lib64/python2.7/site-packages/snappy" directory (snappy installation directory). --> After that i checked python -c 'import snappy; print dir(snappy); print snappy.__file__' [root@localhost snappy]# python -c 'import snappy; print dir(snappy); print snappy.__file__' ['CompressError', 'CompressedLengthError', 'InvalidCompressedInputError', 'StreamCompressor', 'StreamDecompressor', 'UncompressError', '_CHUNK_MAX', '_COMPRESSED_CHUNK', '_COMPRESSION_THRESHOLD', '_IDENTIFIER_CHUNK', '_RESERVED_SKIPPABLE', '_RESERVED_UNSKIPPABLE', '_STREAM_IDENTIFIER', '_STREAM_TO_STREAM_BLOCK_SIZE', '_UNCOMPRESSED_CHUNK', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '_compress', '_crc32c', '_masked_crc32c', '_uncompress', 'cmdline_main', 'compress', 'decompress', 'isValidCompressed', 'stream_compress', 'stream_decompress', 'struct', 'uncompress'] snappy.pyc --> we found the compress and related modules all being present. --> Then i ran ./cqlsh and it ran sucessfully.No More Errors Thanks for your support > AttributeError: 'module' object has no attribute 'compress' > > > Key: CASSANDRA-10255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10255 > Project: Cassandra > Issue Type: Bug > Components: Config, Packaging > Environment: Linux fedora 21 >Reporter: navithejesh > Labels: cqlsh > Fix For: 2.1.x > > Attachments: erroe2.jpeg, error.jpeg > > > Hi Recently i have upgrade my server from cassandra version 2.0.11 to 2.1.8 > using tarball installation.And i have added one node to existing culster with > the same version of 2.1.8. All the configuration settings are done but am not > able to run cqlsh from bin directory.and this is the error > AttributeError: 'module' object has no attribute 'compress' > {code} > ./cqlsh > Traceback (most recent call last): > File "./cqlsh", line 111, in > from cassandra.cluster import Cluster, PagedResult > File > "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py", > line 49, in > File > "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/connection.py", > line 88, in > AttributeError: 'module' object has no attribute 'compress' > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10085) Allow BATCH with conditions to span multiple tables with same partition key
[ https://issues.apache.org/jira/browse/CASSANDRA-10085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733311#comment-14733311 ] Dennis Roppelt edited comment on CASSANDRA-10085 at 9/7/15 7:06 AM: Hello Sylvain, Thank you for the quick and informative reply. I already anticipated that this would not be trivial to implement, and might go against some design principles created in C*. I also can see why this won't be implemented soon or at all. I will look into a different approach of modelling my data or see if another database is a better fit (obviously standard RDBMS will work for the model). It is a bit disappointing though, that this limitation was not mentioned in the documentation. Compared to other NoSQL-DBs, Cassandra usually has an outstandingly good documentation, with lots of examples and details. How would I suggest adding a hint about the limitiation with batch + conditional statements across tables? Creating a new issue and link to this one? was (Author: roppeldidu): Hello Sylvain, Thank you for the quick and informative reply. I already anticipated that this would not be trivial to implement, and might go against some design principles created in C*. I also can see why this won't be implemented soon or at all. I will look into a different approach of modelling my data or see if another database is a better fit (obviously standard RDBMS will work for the model). It is a bit disappointing though, that this limitation was not mentioned in the documentation. Compared to other NoSQL-DBs, Cassandra has an outstandingly good documentation. How would I suggest adding a hint about the limitiation with batch + conditional statements across tables? Creating a new issue and link to this one? > Allow BATCH with conditions to span multiple tables with same partition key > --- > > Key: CASSANDRA-10085 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10085 > Project: Cassandra > Issue Type: Improvement >Reporter: Dennis Roppelt > Labels: batch, conditional-statement, partitioning > > h4. Use case: > I want to use batch to keep data between tables synchronized with help of > conditionals. In particular, I want to have a ONE-to-ONE relationship in my > data model. I dont want to have an unintended upsert while trying to maintain > a consistent one-to-one relationship (write requests from other clients that > try to create a different relationship with one of the keys I inteded to > use), which is why I would like the IF NOT EXISTS when inserting data into > multiple tables in a batch. > But when trying to insert data in a batch-block with an IF NOT EXISTS, I get > the following response (reproduceable example further below): > {code} > cqlsh:testkeyspace> > BEGIN BATCH > INSERT INTO key1_to_key2 (partKey, key1, key2) values (1,2,3) IF NOT EXISTS; > INSERT INTO key2_to_key1 (partKey, key1, key2) values (1,2,3) IF NOT EXISTS; > APPLY BATCH; > Bad Request: Batch with conditions cannot span multiple tables{code} > The tables have a primary key on (partKey, key1) and (partKey, key2). Which > means that partKey in both cases is their partition key. Which in the example > also is the same value. > h4. Why I want to use a BATCH-statement that way: > In traditional databases, to design a one-to-one relationship, I would create > one table with two keys, both with a unique constraint: > {code:sql} > CREATE TABLE `myOneToOneTable` ( > `ID` int NOT NULL, > `KEY_ONE` int NOT NULL, > `KEY_TWO` int NOT NULL, > // some other fields > PRIMARY KEY (`ID`), > UNIQUE KEY `KEY_ONE`, > UNIQUE KEY `KEY_TWO` > ); > {code} > (simplified example taken from an [one-to-one example for > mysql|http://www.mkyong.com/mysql/how-to-define-one-to-one-relationship-in-mysql/]) > As far as I know, unique constraints are not supported in Cassandra, so > another way around it would be to maintain the relationshipt between two > tables, each sharing the same partition key. For which I would like Cassandra > to make sure that when I insert data, either both are inserted at the same > time or none (to prevent other writing client to insert a different > one-to-one relationship than me). > [According to the documentation, this is what BATCH-Statements are used > for|http://docs.datastax.com/en/cql/3.3/cql/cql_using/useBatch.html] > {quote} > Instead, using batches to synchronize data to tables is a legitimate > operation. > {quote} > I expected it to be able to do it, but as seen above, I cannot enforce INSERT > IF NOT EXISTS over multiple tables. > I do understand the background as to *why* there is a limitation coded that > way, so that not too many nodes have to take part in processing of the batch. > However, if the operat
[jira] [Commented] (CASSANDRA-8907) Raise GCInspector alerts to WARN
[ https://issues.apache.org/jira/browse/CASSANDRA-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1470#comment-1470 ] Adam Hattrell commented on CASSANDRA-8907: -- Even for the slackest user requirements - I've never a system that a seconds pause didn't kill it. I would strongly advocate for warning at least at that level. I would probably advocate 500ms as being significantly poor. I've seen a few users that manage at 200ms. I'd like 500ms in current and then 200 in 3.0. I'd settle for 1000 in current. I'd prefer 20k to disabled! > Raise GCInspector alerts to WARN > > > Key: CASSANDRA-8907 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8907 > Project: Cassandra > Issue Type: Improvement >Reporter: Adam Hattrell >Assignee: Amit Singh Chowdhery > Labels: patch > Attachments: cassnadra-8907.patch > > > I'm fairly regularly running into folks wondering why their applications are > reporting down nodes. Yet, they report, when they grepped the logs they have > no WARN or ERRORs listed. > Nine times out of ten, when I look through the logs we see a ton of ParNew or > CMS gc pauses occurring similar to the following: > INFO [ScheduledTasks:1] 2013-03-07 18:44:46,795 GCInspector.java (line 122) > GC for ConcurrentMarkSweep: 1835 ms for 3 collections, 2606015656 used; max > is 10611589120 > INFO [ScheduledTasks:1] 2013-03-07 19:45:08,029 GCInspector.java (line 122) > GC for ParNew: 9866 ms for 8 collections, 2910124308 used; max is 6358564864 > To my mind these should be WARN's as they have the potential to be > significantly impacting the clusters performance as a whole. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10276) With DTCS, do STCS in windows if more than max_threshold sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-10276: Fix Version/s: 2.2.x > With DTCS, do STCS in windows if more than max_threshold sstables > - > > Key: CASSANDRA-10276 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10276 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.x, 2.1.x, 2.2.x > > > To avoid constant recompaction of files in DTCS windows, we should do STCS of > those files. > Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10276) With DTCS, do STCS in windows if more than max_threshold sstables
Marcus Eriksson created CASSANDRA-10276: --- Summary: With DTCS, do STCS in windows if more than max_threshold sstables Key: CASSANDRA-10276 URL: https://issues.apache.org/jira/browse/CASSANDRA-10276 Project: Cassandra Issue Type: Sub-task Reporter: Marcus Eriksson Assignee: Marcus Eriksson To avoid constant recompaction of files in DTCS windows, we should do STCS of those files. Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10276) With DTCS, do STCS in windows if more than max_threshold sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-10276: Description: To avoid constant recompaction of files in big ( > max threshold) DTCS windows, we should do STCS of those files. Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs was: To avoid constant recompaction of files in DTCS windows, we should do STCS of those files. Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs > With DTCS, do STCS in windows if more than max_threshold sstables > - > > Key: CASSANDRA-10276 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10276 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.x, 2.1.x, 2.2.x > > > To avoid constant recompaction of files in big ( > max threshold) DTCS > windows, we should do STCS of those files. > Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733448#comment-14733448 ] Benedict commented on CASSANDRA-8630: - Neither of them are actually bugs. I don't mind if we "fix" them, but we don't _have_ to; just those that are genuine problems. Let me know if you'd still like to commit the patch. > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733463#comment-14733463 ] Stefania commented on CASSANDRA-8630: - {{FB.NS_DANGEROUS_NON_SHORT_CIRCUIT}} I don't care much about either. As for {{FB.UR_UNINIT_READ_CALLED_FROM_SUPER_CONSTRUCTOR}}, even if it's not a problem at the moment, it may confuse somebody in future and cause a problem. I also don't mind either way though, so let's wait for [~aweisberg]'s input as well and then decide? CI: http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-8630-3.0-coverity-dtest/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-8630-3.0-coverity-testall/ A few failed dtests but the failures are also on the unpatched 3.0. > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10274) Assertion Errors when interrupting Cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-10274: - Priority: Critical (was: Major) > Assertion Errors when interrupting Cleanup > -- > > Key: CASSANDRA-10274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10274 > Project: Cassandra > Issue Type: Bug >Reporter: Jeff Jirsa >Priority: Critical > Fix For: 2.1.x > > > Exceptions encountered after Interrupting cleanup on 2.1.7 - logging due to > nature. Seen on 2 different nodes. May be related to CASSANDRA-10260 . > {code} > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,630 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 44698243387/139158 > 624314)bytes > INFO [CompactionExecutor:5838] 2015-09-06 11:33:39,638 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 37886026781/133123 > 638379)bytes > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,639 > CompactionManager.java:749 - Cleaning up > SSTableReader(path='/mnt/cassandra/data/keyspace/table-74bffc100fbe11e5a5ce37a8b36fe285/keyspace-table-26598-Data.db') > ERROR [CompactionExecutor:5838] 2015-09-06 11:33:39,639 > CassandraDaemon.java:223 - Exception in thread > Thread[CompactionExecutor:5838,1,main] > java.lang.AssertionError: Memory was freed > at > org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:97) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at org.apache.cassandra.io.util.Memory.getInt(Memory.java:281) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:139) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:144) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:113) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:1183) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.firstKeyBeyond(SSTableReader.java:1667) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.needsCleanup(CompactionManager.java:693) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.doCleanupOne(CompactionManager.java:734) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:94) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:389) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:285) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_51] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > {code} > And: > {code} > ERROR [IndexSummaryManager:1] 2015-09-06 09:32:23,346 > CassandraDaemon.java:223 - Exception in thread > Thread[IndexSummaryManager:1,1,main] > java.lang.AssertionError: null > at > org.apache.cassandra.io.sstable.SSTableReader.setReplacedBy(SSTableReader.java:955) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneAndReplace(SSTableReader.java:1002) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneWithNewSummarySamplingLevel(SSTableReader.java:1105) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.adjustSamplingLevels(IndexSummaryManager.java:421) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:299) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.utils.WrappedRunnab
[jira] [Commented] (CASSANDRA-10209) Missing role manager in cassandra.yaml causes unexpected behaviour
[ https://issues.apache.org/jira/browse/CASSANDRA-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733554#comment-14733554 ] Sam Tunnicliffe commented on CASSANDRA-10209: - bq. I'm not sure what other resources should be excluded from client mode, and I'd rather not do it halfway. Not sure I completely follow; it isn't that those resources are excluded from client mode, rather that when *not* in client mode (and so cassandra.yaml isn't read), don't attempt to figure out which of the {{system_auth}} tables are not modifiable. Which seems reasonable, as if an IAuthenticator/IAuthorizer/IRoleManager is set in the yaml, those won't be correct anyway. > Missing role manager in cassandra.yaml causes unexpected behaviour > -- > > Key: CASSANDRA-10209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10209 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Minor > Fix For: 2.2.x, 3.0.0 rc1 > > > On upgrading to 2.2+, if the new {{role_manager}} option is not added to > {{cassandra.yaml}}, an instance of the default {{CassandraRoleManager}} is > created during initialization of {{DatabaseDescriptor}}. This is a problem as > the set of role options supported by {{CRM}} depends on the configured > {{IAuthenticator}}, which at that point in time is always > {{AllowAllAuthenticator}}. > This StackOverflow post describes the problem; the configured authenticator > is {{PasswordAuthenticator}}, the role manager should allow roles to be > created using the {{PASSWORD}} option, but it does not. > http://stackoverflow.com/questions/31820914/in-cassandra-2-2-unable-to-create-role-containing-password > The simple workaround is to ensure that yaml contains the role manager option > {code} > role_manager: CassandraRoleManager > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10274) Assertion Errors when interrupting Cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733566#comment-14733566 ] Benedict commented on CASSANDRA-10274: -- Patch available [here|https://github.com/belliottsmith/cassandra/tree/10274-2.1]. I've also pushed a merge for 2.2 [here|https://github.com/belliottsmith/cassandra/tree/10274-2.2]. 2.2 isn't actually affected by this bug, but it conflicts with changes that fixed it, and in merging I used our newish {{Throwables}} static utilities. > Assertion Errors when interrupting Cleanup > -- > > Key: CASSANDRA-10274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10274 > Project: Cassandra > Issue Type: Bug >Reporter: Jeff Jirsa >Priority: Critical > Fix For: 2.1.x > > > Exceptions encountered after Interrupting cleanup on 2.1.7 - logging due to > nature. Seen on 2 different nodes. May be related to CASSANDRA-10260 . > {code} > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,630 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 44698243387/139158 > 624314)bytes > INFO [CompactionExecutor:5838] 2015-09-06 11:33:39,638 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 37886026781/133123 > 638379)bytes > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,639 > CompactionManager.java:749 - Cleaning up > SSTableReader(path='/mnt/cassandra/data/keyspace/table-74bffc100fbe11e5a5ce37a8b36fe285/keyspace-table-26598-Data.db') > ERROR [CompactionExecutor:5838] 2015-09-06 11:33:39,639 > CassandraDaemon.java:223 - Exception in thread > Thread[CompactionExecutor:5838,1,main] > java.lang.AssertionError: Memory was freed > at > org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:97) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at org.apache.cassandra.io.util.Memory.getInt(Memory.java:281) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:139) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:144) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:113) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:1183) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.firstKeyBeyond(SSTableReader.java:1667) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.needsCleanup(CompactionManager.java:693) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.doCleanupOne(CompactionManager.java:734) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:94) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:389) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:285) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_51] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > {code} > And: > {code} > ERROR [IndexSummaryManager:1] 2015-09-06 09:32:23,346 > CassandraDaemon.java:223 - Exception in thread > Thread[IndexSummaryManager:1,1,main] > java.lang.AssertionError: null > at > org.apache.cassandra.io.sstable.SSTableReader.setReplacedBy(SSTableReader.java:955) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneAndReplace(SSTableReader.java:1002) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneWithNewSummarySamplingLevel(SSTableReader.java:1105) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.adjustSamplingLevels(IndexSummaryManager.java:421) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:299) > ~[apache-cassandra-2.1.7.jar:2.1.7] >
[jira] [Commented] (CASSANDRA-10146) Deprecate v1 and v2 protocol in 2.2, drop support in 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733588#comment-14733588 ] Benjamin Lerer commented on CASSANDRA-10146: The new patch for 2.2 is [here|https://github.com/apache/cassandra/compare/trunk...blerer:10146-2.2] and the new one for 3.0 is [here|https://github.com/apache/cassandra/compare/trunk...blerer:10146-3.0] * the unit test results for 2.2 are [here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-2.2-testall/5/] * the dtest results for 2.2 are [here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-2.2-dtest/4/] * the unit test results for 3.0 are [here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-3.0-testall/3/] * the dtest results for 3.0 are [here|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-10146-3.0-dtest/7/] > Deprecate v1 and v2 protocol in 2.2, drop support in 3.0 > > > Key: CASSANDRA-10146 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10146 > Project: Cassandra > Issue Type: Task > Components: Core >Reporter: Tyler Hobbs >Assignee: Benjamin Lerer > Labels: client-impacting, doc-impacting > Fix For: 2.2.x, 3.0.0 rc1 > > > In 3.0, we would like to use frozen collections in the system keyspaces, and > it seems likely that we will eventually want to use tuples or nested > collections as well. Drivers that only support protocol versions 1 and 2 > will not be able to read these system keyspaces because they cannot decode > those types. > I think this is a good time to start deprecating and dropping support for old > protocol versions. The v3 protocol was introduced in 2.1, so if we remove > support for v1 and v2 in 3.0, that gives users two major versions to upgrade > their drivers. Fortunately, upgrading drivers to a version that supports the > v3 protocol is generally straightforward. > The benefits of doing this are: > * We can use new types in the system keyspaces > * We can eliminate protocol-version-specific encoding and decoding of > collections within Cassandra > * Driver maintainers can eventually drop support for old protocol versions > once all C* versions that support them are EOL > To avoid a hard drop of v1 and v2 support in 3.0, I propose that we also > officially deprecate these in 2.2. Unfortunately, we don't have > protocol-level warnings until v4, so we can't use that to notify users of the > deprecation, but the combination of a NEWS.txt entry, warning logs, and > (potentially) driver-level warnings should suffice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10215) Reduce redundant secondary index selection lookups
[ https://issues.apache.org/jira/browse/CASSANDRA-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733598#comment-14733598 ] Sam Tunnicliffe commented on CASSANDRA-10215: - bq. Why use a UUID vs just passing the name? Actually, my intention was to keep the size bounded, more than just to have it fixed. Do you think the additional debugging info is worth the potential increase in message size? It's probably only a few bytes in most cases, though I would guess that most index names are > 16 bytes. bq. Why do you have IndexMetadata in the constructor of the read command when it's always passing Optional.empty() in the code? It wasn't *always* passing {{Optional.empty()}}, the super call from {{PartitionRangeReadCommand}} would pass through its own argument which may not be empty. You're right though, originally the member was final but now the {{ReadCommand}} ctor doesn't need that argument, as PRRC can just set it in its own ctor. I've pushed a commit to fix that up. > Reduce redundant secondary index selection lookups > -- > > Key: CASSANDRA-10215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10215 > Project: Cassandra > Issue Type: Improvement >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe > Fix For: 3.0.0 rc1 > > > Once an index has been identified as being the one that should be used for a > given query, it should be included as part of the {{ReadCommand}} so that it > can be retrieved when needed during execution. Currently we don't do this, > meaning that the index selection process implemented in > {{SIM::getBestIndexFor}} is run multiple times on both the coordinator and > each replica when performing a range command. > This is also a pre-requisite of CASSANDRA-10214 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8907) Raise GCInspector alerts to WARN
[ https://issues.apache.org/jira/browse/CASSANDRA-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733756#comment-14733756 ] Anuj Wadehra commented on CASSANDRA-8907: - I am adding a 3rd scenario to the 2 scenarios I mentioned in my earlier comment: 3. GC warn threshold is enabled by default and set to 5000ms.Suppose an application is NOT sensitive to gc pauses e.g. some background job. Even though no functionality is impacted and application SLA is being met, its getting 5+ secs of gc pauses in the background. When the user upgrades Cassandra he will start getting Warnings for every gc pause over 5 sec. I wont call that 'breaking' of existing log monitoring system with new warnings. Warnings are warnings "an indication of possible problem" not errors. Any gc pause over 5 secs indicates poor heap tuning / insufficient heap. After upgrade, User must start getting these warnings so that he can look at options for optimizing JVM tunings. Based on the 3 scenarios I mentioned, scenarios 2 and 3 support enabling this property by default and setting value to something like 5+ secs so that user is aware of possible problems with GC tuning upfront. If user is warned and he still wants to continue with long gc pauses, he can increase the gc warn threshold. But its should be Cassandra's responsibility to make user aware of possible problems by raising warning especially when we have a GCInspector which is monitoring Gc pauses. > Raise GCInspector alerts to WARN > > > Key: CASSANDRA-8907 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8907 > Project: Cassandra > Issue Type: Improvement >Reporter: Adam Hattrell >Assignee: Amit Singh Chowdhery > Labels: patch > Attachments: cassnadra-8907.patch > > > I'm fairly regularly running into folks wondering why their applications are > reporting down nodes. Yet, they report, when they grepped the logs they have > no WARN or ERRORs listed. > Nine times out of ten, when I look through the logs we see a ton of ParNew or > CMS gc pauses occurring similar to the following: > INFO [ScheduledTasks:1] 2013-03-07 18:44:46,795 GCInspector.java (line 122) > GC for ConcurrentMarkSweep: 1835 ms for 3 collections, 2606015656 used; max > is 10611589120 > INFO [ScheduledTasks:1] 2013-03-07 19:45:08,029 GCInspector.java (line 122) > GC for ParNew: 9866 ms for 8 collections, 2910124308 used; max is 6358564864 > To my mind these should be WARN's as they have the potential to be > significantly impacting the clusters performance as a whole. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10277) Empty BATCH throws exception in Cassandra
Vovodroid created CASSANDRA-10277: - Summary: Empty BATCH throws exception in Cassandra Key: CASSANDRA-10277 URL: https://issues.apache.org/jira/browse/CASSANDRA-10277 Project: Cassandra Issue Type: Bug Components: Core Reporter: Vovodroid Priority: Minor Attachments: emptybatch.patch Connect to Cassandra with cqlsh and perform empty BATCH statement: {code}BEGIN BATCH APPLY BATCH;{code} Following exception is thrown and logged {code} ERROR 14:02:07 Unexpected exception during request java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_45] at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_45] at org.apache.cassandra.cql3.statements.BatchStatement$Parsed.prepare(BatchStatement.java:514) ~[main/:na] ... {code} It's caused because of access to batch statement list, even if it's empty. Proposed patch (for 2.2.0) is attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10209) Missing role manager in cassandra.yaml causes unexpected behaviour
[ https://issues.apache.org/jira/browse/CASSANDRA-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733794#comment-14733794 ] Jonathan Ellis commented on CASSANDRA-10209: Do you want to recommit with Carl's patch then? > Missing role manager in cassandra.yaml causes unexpected behaviour > -- > > Key: CASSANDRA-10209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10209 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Minor > Fix For: 2.2.x, 3.0.0 rc1 > > > On upgrading to 2.2+, if the new {{role_manager}} option is not added to > {{cassandra.yaml}}, an instance of the default {{CassandraRoleManager}} is > created during initialization of {{DatabaseDescriptor}}. This is a problem as > the set of role options supported by {{CRM}} depends on the configured > {{IAuthenticator}}, which at that point in time is always > {{AllowAllAuthenticator}}. > This StackOverflow post describes the problem; the configured authenticator > is {{PasswordAuthenticator}}, the role manager should allow roles to be > created using the {{PASSWORD}} option, but it does not. > http://stackoverflow.com/questions/31820914/in-cassandra-2-2-unable-to-create-role-containing-password > The simple workaround is to ensure that yaml contains the role manager option > {code} > role_manager: CassandraRoleManager > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10278) CQLSH version is not supported by Remote in CASSANDRA 2.1.9
Puneet Pant created CASSANDRA-10278: --- Summary: CQLSH version is not supported by Remote in CASSANDRA 2.1.9 Key: CASSANDRA-10278 URL: https://issues.apache.org/jira/browse/CASSANDRA-10278 Project: Cassandra Issue Type: Bug Components: Core Reporter: Puneet Pant Priority: Critical Fix For: 2.1.x Although the sever run successfully as: INFO 14:27:31 Starting listening for CQL clients on localhost/127.0.0.1:9046... INFO 14:27:31 Binding thrift service to localhost/127.0.0.1:9160 INFO 14:27:31 Listening for thrift clients... But CQLSH can not connect to it. This is because the shipped version of CQLSH with cassandra is 3.2.0 whereas the supported by remote is 3.3.0 so the default shipped version of CQL need to be upgraded in cassandra or it should provide backward compatibility. Here is the CQLSH error: Connection error: ('Unable to connect to any servers', {'127.0.0.1': ProtocolError("cql_version '3.2.0' is not supported by remote (w/ native protocol). Supported versions: [u'3.3.0']",)}) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10278) CQLSH version is not supported by Remote in CASSANDRA 2.1.9
[ https://issues.apache.org/jira/browse/CASSANDRA-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Puneet Pant updated CASSANDRA-10278: Description: Although the CASSANDRA 2.1.9 sever run successfully as: INFO 14:27:31 Starting listening for CQL clients on localhost/127.0.0.1:9046... INFO 14:27:31 Binding thrift service to localhost/127.0.0.1:9160 INFO 14:27:31 Listening for thrift clients... But CQLSH can not connect to it. This is because the shipped version of CQLSH with cassandra is 3.2.0 whereas the supported by remote is 3.3.0 so the default shipped version of CQL need to be upgraded in cassandra or it should provide backward compatibility. Here is the CQLSH error: Connection error: ('Unable to connect to any servers', {'127.0.0.1': ProtocolError("cql_version '3.2.0' is not supported by remote (w/ native protocol). Supported versions: [u'3.3.0']",)}) was: Although the sever run successfully as: INFO 14:27:31 Starting listening for CQL clients on localhost/127.0.0.1:9046... INFO 14:27:31 Binding thrift service to localhost/127.0.0.1:9160 INFO 14:27:31 Listening for thrift clients... But CQLSH can not connect to it. This is because the shipped version of CQLSH with cassandra is 3.2.0 whereas the supported by remote is 3.3.0 so the default shipped version of CQL need to be upgraded in cassandra or it should provide backward compatibility. Here is the CQLSH error: Connection error: ('Unable to connect to any servers', {'127.0.0.1': ProtocolError("cql_version '3.2.0' is not supported by remote (w/ native protocol). Supported versions: [u'3.3.0']",)}) > CQLSH version is not supported by Remote in CASSANDRA 2.1.9 > --- > > Key: CASSANDRA-10278 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10278 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Puneet Pant >Priority: Critical > Fix For: 2.1.x > > > Although the CASSANDRA 2.1.9 sever run successfully as: > INFO 14:27:31 Starting listening for CQL clients on > localhost/127.0.0.1:9046... > INFO 14:27:31 Binding thrift service to localhost/127.0.0.1:9160 > INFO 14:27:31 Listening for thrift clients... > But CQLSH can not connect to it. This is because the shipped version of > CQLSH with cassandra is 3.2.0 whereas the supported by remote is 3.3.0 so the > default shipped version of CQL need to be upgraded in cassandra or it should > provide backward compatibility. > Here is the CQLSH error: > Connection error: ('Unable to connect to any servers', {'127.0.0.1': > ProtocolError("cql_version '3.2.0' is not supported by remote (w/ native > protocol). Supported versions: [u'3.3.0']",)}) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10278) CQLSH version is not supported by Remote in CASSANDRA 2.1.9
[ https://issues.apache.org/jira/browse/CASSANDRA-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Puneet Pant updated CASSANDRA-10278: Description: Although the CASSANDRA 2.1.9 sever runs successfully as: INFO 14:27:31 Starting listening for CQL clients on localhost/127.0.0.1:9046... INFO 14:27:31 Binding thrift service to localhost/127.0.0.1:9160 INFO 14:27:31 Listening for thrift clients... But CQLSH can not connect to it. This is because the shipped version of CQLSH with cassandra is 3.2.0 whereas the supported by remote is 3.3.0 so the default shipped version of CQL need to be upgraded in cassandra or it should provide backward compatibility. Here is the CQLSH error: Connection error: ('Unable to connect to any servers', {'127.0.0.1': ProtocolError("cql_version '3.2.0' is not supported by remote (w/ native protocol). Supported versions: [u'3.3.0']",)}) Note:9046 is just changed to fix conflicts. was: Although the CASSANDRA 2.1.9 sever run successfully as: INFO 14:27:31 Starting listening for CQL clients on localhost/127.0.0.1:9046... INFO 14:27:31 Binding thrift service to localhost/127.0.0.1:9160 INFO 14:27:31 Listening for thrift clients... But CQLSH can not connect to it. This is because the shipped version of CQLSH with cassandra is 3.2.0 whereas the supported by remote is 3.3.0 so the default shipped version of CQL need to be upgraded in cassandra or it should provide backward compatibility. Here is the CQLSH error: Connection error: ('Unable to connect to any servers', {'127.0.0.1': ProtocolError("cql_version '3.2.0' is not supported by remote (w/ native protocol). Supported versions: [u'3.3.0']",)}) > CQLSH version is not supported by Remote in CASSANDRA 2.1.9 > --- > > Key: CASSANDRA-10278 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10278 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Puneet Pant >Priority: Critical > Fix For: 2.1.x > > > Although the CASSANDRA 2.1.9 sever runs successfully as: > INFO 14:27:31 Starting listening for CQL clients on > localhost/127.0.0.1:9046... > INFO 14:27:31 Binding thrift service to localhost/127.0.0.1:9160 > INFO 14:27:31 Listening for thrift clients... > But CQLSH can not connect to it. This is because the shipped version of > CQLSH with cassandra is 3.2.0 whereas the supported by remote is 3.3.0 so the > default shipped version of CQL need to be upgraded in cassandra or it should > provide backward compatibility. > Here is the CQLSH error: > Connection error: ('Unable to connect to any servers', {'127.0.0.1': > ProtocolError("cql_version '3.2.0' is not supported by remote (w/ native > protocol). Supported versions: [u'3.3.0']",)}) > Note:9046 is just changed to fix conflicts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10209) Missing role manager in cassandra.yaml causes unexpected behaviour
[ https://issues.apache.org/jira/browse/CASSANDRA-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733815#comment-14733815 ] Sam Tunnicliffe commented on CASSANDRA-10209: - Sure, just wanted to check I wasn't misunderstanding what you meant > Missing role manager in cassandra.yaml causes unexpected behaviour > -- > > Key: CASSANDRA-10209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10209 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Minor > Fix For: 2.2.x, 3.0.0 rc1 > > > On upgrading to 2.2+, if the new {{role_manager}} option is not added to > {{cassandra.yaml}}, an instance of the default {{CassandraRoleManager}} is > created during initialization of {{DatabaseDescriptor}}. This is a problem as > the set of role options supported by {{CRM}} depends on the configured > {{IAuthenticator}}, which at that point in time is always > {{AllowAllAuthenticator}}. > This StackOverflow post describes the problem; the configured authenticator > is {{PasswordAuthenticator}}, the role manager should allow roles to be > created using the {{PASSWORD}} option, but it does not. > http://stackoverflow.com/questions/31820914/in-cassandra-2-2-unable-to-create-role-containing-password > The simple workaround is to ensure that yaml contains the role manager option > {code} > role_manager: CassandraRoleManager > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9761) Delay auth setup until peers are upgraded
[ https://issues.apache.org/jira/browse/CASSANDRA-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733856#comment-14733856 ] Sylvain Lebresne commented on CASSANDRA-9761: - For info, this is the reason for the failure of at least some upgrade dtests (typically [this|http://cassci.datastax.com/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/upgrade_tests.paging_test/TestPagingData/test_paging_a_single_wide_row/]). Basically, the test are issuing a truncate as their first order of business, and the 2.1 closes the connection to the other node due to this, some of the truncation acknowledgment get losts because it's in the queue of that connection, hence ending up in a truncate timeout. And of course there is the fact that due to this the 2.1 node logs a warning with a stack trace, which might worry operators a bit even though nothing is wrong. > Delay auth setup until peers are upgraded > - > > Key: CASSANDRA-9761 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9761 > Project: Cassandra > Issue Type: Improvement >Reporter: Sam Tunnicliffe > Fix For: 3.0.0 rc1, 2.2.2 > > > The built in auth classes {{CassandraRoleManager}} and > {{CassandraAuthorizer}} both attempt to do some setup and data conversion > when a node is upgraded to version 2.2 or higher. At the moment, each node > attempts the operations with the expectation that this will fail until enough > of the cluster has been upgraded for it to succeed (i.e. enough nodes have > the latest schema with the requisite new tables). These expected failures are > largely harmless, but they are annoying because they cause the receiving node > (the non-upgraded node) to close the connection with the upgraded node, which > then has to be restablished. Although this is the normal behaviour on schema > disagreement (see CASSANDRA-9136 for further discussion), it may be possible > to avoid in this specific circumstance. Given that we expect the operations > to fail until enough nodes are upgraded, we could defer them until we're sure > they can succeed by checking the messaging service version of peers. > Right now these are a one shot thing, each node only makes one attempt at the > conversion (until it is restarted). Without investigating further, I don't > know if we'd need to add in retries in case it takes a little time for each > peer's MS version to be updated as they're upgraded. The setup & conversion > operations are idempotent, so there shouldn't be a great issue if several > nodes attempt them at the same time anyway. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9761) Delay auth setup until peers are upgraded
[ https://issues.apache.org/jira/browse/CASSANDRA-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-9761: Fix Version/s: 2.2.2 3.0.0 rc1 > Delay auth setup until peers are upgraded > - > > Key: CASSANDRA-9761 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9761 > Project: Cassandra > Issue Type: Improvement >Reporter: Sam Tunnicliffe > Fix For: 3.0.0 rc1, 2.2.2 > > > The built in auth classes {{CassandraRoleManager}} and > {{CassandraAuthorizer}} both attempt to do some setup and data conversion > when a node is upgraded to version 2.2 or higher. At the moment, each node > attempts the operations with the expectation that this will fail until enough > of the cluster has been upgraded for it to succeed (i.e. enough nodes have > the latest schema with the requisite new tables). These expected failures are > largely harmless, but they are annoying because they cause the receiving node > (the non-upgraded node) to close the connection with the upgraded node, which > then has to be restablished. Although this is the normal behaviour on schema > disagreement (see CASSANDRA-9136 for further discussion), it may be possible > to avoid in this specific circumstance. Given that we expect the operations > to fail until enough nodes are upgraded, we could defer them until we're sure > they can succeed by checking the messaging service version of peers. > Right now these are a one shot thing, each node only makes one attempt at the > conversion (until it is restarted). Without investigating further, I don't > know if we'd need to add in retries in case it takes a little time for each > peer's MS version to be updated as they're upgraded. The setup & conversion > operations are idempotent, so there shouldn't be a great issue if several > nodes attempt them at the same time anyway. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9761) Delay auth setup until peers are upgraded
[ https://issues.apache.org/jira/browse/CASSANDRA-9761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-9761: Assignee: Sam Tunnicliffe > Delay auth setup until peers are upgraded > - > > Key: CASSANDRA-9761 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9761 > Project: Cassandra > Issue Type: Improvement >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe > Fix For: 3.0.0 rc1, 2.2.2 > > > The built in auth classes {{CassandraRoleManager}} and > {{CassandraAuthorizer}} both attempt to do some setup and data conversion > when a node is upgraded to version 2.2 or higher. At the moment, each node > attempts the operations with the expectation that this will fail until enough > of the cluster has been upgraded for it to succeed (i.e. enough nodes have > the latest schema with the requisite new tables). These expected failures are > largely harmless, but they are annoying because they cause the receiving node > (the non-upgraded node) to close the connection with the upgraded node, which > then has to be restablished. Although this is the normal behaviour on schema > disagreement (see CASSANDRA-9136 for further discussion), it may be possible > to avoid in this specific circumstance. Given that we expect the operations > to fail until enough nodes are upgraded, we could defer them until we're sure > they can succeed by checking the messaging service version of peers. > Right now these are a one shot thing, each node only makes one attempt at the > conversion (until it is restarted). Without investigating further, I don't > know if we'd need to add in retries in case it takes a little time for each > peer's MS version to be updated as they're upgraded. The setup & conversion > operations are idempotent, so there shouldn't be a great issue if several > nodes attempt them at the same time anyway. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10279) Inconsistent update/delete behavior for static lists vs lists
Lex Lythius created CASSANDRA-10279: --- Summary: Inconsistent update/delete behavior for static lists vs lists Key: CASSANDRA-10279 URL: https://issues.apache.org/jira/browse/CASSANDRA-10279 Project: Cassandra Issue Type: Bug Components: Core Environment: [cqlsh 5.0.1 | Cassandra 2.2.0 | CQL spec 3.3.0 | Native protocol v4] Ubuntu 14.04 x64 Reporter: Lex Lythius Partial list deletions (either in the form of {{UPDATE list = list - \[...]}} or {{DELETE list\[index]}} form) work fine in regular list columns, but do nothing or throw bad index error when applied to *static* list columns. Example: {code:sql} create table t_lists ( id int, dt int, ls list, sls list static, primary key(id, dt) ); cqlsh:test> update t_lists set ls = ['foo', 'bar', 'baz'], sls = ['chico', 'harpo', 'zeppo', 'groucho'] where id=1 and dt=1; cqlsh:test> select * from t_lists; id | dt | sls| ls +++--- 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo', 'bar', 'baz'] (1 rows) cqlsh:test> delete ls[2] from t_lists where id = 1 and dt = 1; -- works cqlsh:test> delete sls[2] from t_lists where id = 1; -- errors InvalidRequest: code=2200 [Invalid query] message="Attempted to delete an element from a list which is null" cqlsh:test> select * from t_lists; id | dt | sls| ls +++ 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo', 'bar'] (1 rows) cqlsh:test> update t_lists set ls = ls - ['bar'] where id=1 and dt=1; -- works cqlsh:test> update t_lists set sls = sls - ['zeppo'] where id=1; -- fails silently cqlsh:test> select * from t_lists; id | dt | sls| ls +++- 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo'] (1 rows) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10279) Inconsistent update/delete behavior for static lists vs lists
[ https://issues.apache.org/jira/browse/CASSANDRA-10279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733873#comment-14733873 ] Lex Lythius commented on CASSANDRA-10279: - Maybe the underlying issue is the same as in https://issues.apache.org/jira/browse/CASSANDRA-9838. > Inconsistent update/delete behavior for static lists vs lists > - > > Key: CASSANDRA-10279 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10279 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: [cqlsh 5.0.1 | Cassandra 2.2.0 | CQL spec 3.3.0 | Native > protocol v4] > Ubuntu 14.04 x64 >Reporter: Lex Lythius > > Partial list deletions (either in the form of {{UPDATE list = list - \[...]}} > or {{DELETE list\[index]}} form) work fine in regular list columns, but do > nothing or throw bad index error when applied to *static* list columns. > Example: > {code:sql} > create table t_lists ( > id int, > dt int, > ls list, > sls list static, > primary key(id, dt) > ); > cqlsh:test> update t_lists set ls = ['foo', 'bar', 'baz'], sls = ['chico', > 'harpo', 'zeppo', 'groucho'] where id=1 and dt=1; > cqlsh:test> select * from t_lists; > id | dt | sls| ls > +++--- > 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo', 'bar', 'baz'] > (1 rows) > cqlsh:test> delete ls[2] from t_lists where id = 1 and dt = 1; -- works > cqlsh:test> delete sls[2] from t_lists where id = 1; -- errors > InvalidRequest: code=2200 [Invalid query] message="Attempted to delete an > element from a list which is null" > cqlsh:test> select * from t_lists; > id | dt | sls| ls > +++ > 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo', 'bar'] > (1 rows) > cqlsh:test> update t_lists set ls = ls - ['bar'] where id=1 and dt=1; -- works > cqlsh:test> update t_lists set sls = sls - ['zeppo'] where id=1; -- fails > silently > cqlsh:test> select * from t_lists; > id | dt | sls| ls > +++- > 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo'] > (1 rows) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10279) Inconsistent update/delete behavior for static lists vs lists
[ https://issues.apache.org/jira/browse/CASSANDRA-10279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733873#comment-14733873 ] Lex Lythius edited comment on CASSANDRA-10279 at 9/7/15 4:10 PM: - Most likely the underlying issue is the same as in https://issues.apache.org/jira/browse/CASSANDRA-9838. Removing elements by setting to null doesn't work either: {code:sql} update t_lists set sls[0]=null where id=1; InvalidRequest: code=2200 [Invalid query] message="Attempted to set an element on a list which is null" {code} Said bug entry shows there's a patch available, due to be fixed at 2.1.x. Not fixed in 2.2.0, however. was (Author: lexlythius): Maybe the underlying issue is the same as in https://issues.apache.org/jira/browse/CASSANDRA-9838. > Inconsistent update/delete behavior for static lists vs lists > - > > Key: CASSANDRA-10279 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10279 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: [cqlsh 5.0.1 | Cassandra 2.2.0 | CQL spec 3.3.0 | Native > protocol v4] > Ubuntu 14.04 x64 >Reporter: Lex Lythius > > Partial list deletions (either in the form of {{UPDATE list = list - \[...]}} > or {{DELETE list\[index]}} form) work fine in regular list columns, but do > nothing or throw bad index error when applied to *static* list columns. > Example: > {code:sql} > create table t_lists ( > id int, > dt int, > ls list, > sls list static, > primary key(id, dt) > ); > cqlsh:test> update t_lists set ls = ['foo', 'bar', 'baz'], sls = ['chico', > 'harpo', 'zeppo', 'groucho'] where id=1 and dt=1; > cqlsh:test> select * from t_lists; > id | dt | sls| ls > +++--- > 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo', 'bar', 'baz'] > (1 rows) > cqlsh:test> delete ls[2] from t_lists where id = 1 and dt = 1; -- works > cqlsh:test> delete sls[2] from t_lists where id = 1; -- errors > InvalidRequest: code=2200 [Invalid query] message="Attempted to delete an > element from a list which is null" > cqlsh:test> select * from t_lists; > id | dt | sls| ls > +++ > 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo', 'bar'] > (1 rows) > cqlsh:test> update t_lists set ls = ls - ['bar'] where id=1 and dt=1; -- works > cqlsh:test> update t_lists set sls = sls - ['zeppo'] where id=1; -- fails > silently > cqlsh:test> select * from t_lists; > id | dt | sls| ls > +++- > 1 | 1 | ['chico', 'harpo', 'zeppo', 'groucho'] | ['foo'] > (1 rows) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10280) Make DTCS work well with old data
Marcus Eriksson created CASSANDRA-10280: --- Summary: Make DTCS work well with old data Key: CASSANDRA-10280 URL: https://issues.apache.org/jira/browse/CASSANDRA-10280 Project: Cassandra Issue Type: Sub-task Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 3.x, 2.1.x, 2.2.x Operational tasks become incredibly expensive if you keep around a long timespan of data with DTCS - with default settings and 1 year of data, the oldest window covers about 180 days. Bootstrapping a node with vnodes with this data layout will force cassandra to compact very many sstables in this window. We should probably put a cap on how big the biggest windows can get. We could probably default this to something sane based on max_sstable_age (ie, say we can reasonably handle 1000 sstables per node, then we can make calculate how big the windows should be to allow that) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-9597) DTCS should consider file SIZE in addition to time windowing
[ https://issues.apache.org/jira/browse/CASSANDRA-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson resolved CASSANDRA-9597. Resolution: Duplicate > DTCS should consider file SIZE in addition to time windowing > > > Key: CASSANDRA-9597 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9597 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Priority: Minor > Labels: dtcs > > DTCS seems to work well for the typical use case - writing data in perfect > time order, compacting recent files, and ignoring older files. > However, there are "normal" operational actions where DTCS will fall behind > and is unlikely to recover. > An example of this is streaming operations (for example, bootstrap or loading > data into a cluster using sstableloader), where lots (tens of thousands) of > very small sstables can be created spanning multiple time buckets. In these > case, even if max_sstable_age_days is extended to allow the older incoming > files to be compacted, the selection logic is likely to re-compact large > files with fewer small files over and over, rather than prioritizing > selection of max_threshold smallest files to decrease the number of candidate > sstables as quickly as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8371) DateTieredCompactionStrategy is always compacting
[ https://issues.apache.org/jira/browse/CASSANDRA-8371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson resolved CASSANDRA-8371. Resolution: Duplicate Closing this as duplicate, hoping that this will be resolved in CASSANDRA-9644 > DateTieredCompactionStrategy is always compacting > -- > > Key: CASSANDRA-8371 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8371 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: mck >Assignee: Björn Hegerfors > Labels: compaction, dtcs, performance > Attachments: java_gc_counts_rate-month.png, > read-latency-recommenders-adview.png, read-latency.png, > sstables-recommenders-adviews.png, sstables.png, vg2_iad-month.png > > > Running 2.0.11 and having switched a table to > [DTCS|https://issues.apache.org/jira/browse/CASSANDRA-6602] we've seen that > disk IO and gc count increase, along with the number of reads happening in > the "compaction" hump of cfhistograms. > Data, and generally performance, looks good, but compactions are always > happening, and pending compactions are building up. > The schema for this is > {code}CREATE TABLE search ( > loginid text, > searchid timeuuid, > description text, > searchkey text, > searchurl text, > PRIMARY KEY ((loginid), searchid) > );{code} > We're sitting on about 82G (per replica) across 6 nodes in 4 DCs. > CQL executed against this keyspace, and traffic patterns, can be seen in > slides 7+8 of https://prezi.com/b9-aj6p2esft/ > Attached are sstables-per-read and read-latency graphs from cfhistograms, and > screenshots of our munin graphs as we have gone from STCS, to LCS (week ~44), > to DTCS (week ~46). > These screenshots are also found in the prezi on slides 9-11. > [~pmcfadin], [~Bj0rn], > Can this be a consequence of occasional deleted rows, as is described under > (3) in the description of CASSANDRA-6602 ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10253) Incremental repairs not working as expected with DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson resolved CASSANDRA-10253. - Resolution: Duplicate Fix Version/s: (was: 2.1.x) Closing this as a duplicate of CASSANDRA-9644 - hoping all these issues will be fixed there > Incremental repairs not working as expected with DTCS > - > > Key: CASSANDRA-10253 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10253 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Pre-prod >Reporter: vijay >Assignee: Marcus Eriksson > Labels: dtcs > Attachments: sstablemetadata-cluster-logs.zip, systemfiles 2.zip > > > HI, > we are ingesting data 6 million records every 15 mins into one DTCS table and > relaying on Cassandra for purging the data.Table Schema given below, Issue 1: > we are expecting to see table sstable created on day d1 will not be compacted > after d1 how we are not seeing this, how ever i see some data being purged at > random intervals > Issue 2: when we run incremental repair using "nodetool repair keyspace table > -inc -pr" each sstable is splitting up to multiple smaller SStables and > increasing the total storage.This behavior is same running repairs on any > node and any number of times > There are mutation drop's in the cluster > Table: > {code} > CREATE TABLE TableA ( > F1 text, > F2 int, > createts bigint, > stats blob, > PRIMARY KEY ((F1,F2), createts) > ) WITH CLUSTERING ORDER BY (createts DESC) > AND bloom_filter_fp_chance = 0.01 > AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' > AND comment = '' > AND compaction = {'min_threshold': '12', 'max_sstable_age_days': '1', > 'base_time_seconds': '50', 'class': > 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'} > AND compression = {'sstable_compression': > 'org.apache.cassandra.io.compress.LZ4Compressor'} > AND dclocal_read_repair_chance = 0.0 > AND default_time_to_live = 93600 > AND gc_grace_seconds = 3600 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.0 > AND speculative_retry = '99.0PERCENTILE'; > {code} > Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10274) Assertion Errors when interrupting Cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-10274: Assignee: Benedict Reviewer: Marcus Eriksson > Assertion Errors when interrupting Cleanup > -- > > Key: CASSANDRA-10274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10274 > Project: Cassandra > Issue Type: Bug >Reporter: Jeff Jirsa >Assignee: Benedict >Priority: Critical > Fix For: 2.1.x > > > Exceptions encountered after Interrupting cleanup on 2.1.7 - logging due to > nature. Seen on 2 different nodes. May be related to CASSANDRA-10260 . > {code} > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,630 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 44698243387/139158 > 624314)bytes > INFO [CompactionExecutor:5838] 2015-09-06 11:33:39,638 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 37886026781/133123 > 638379)bytes > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,639 > CompactionManager.java:749 - Cleaning up > SSTableReader(path='/mnt/cassandra/data/keyspace/table-74bffc100fbe11e5a5ce37a8b36fe285/keyspace-table-26598-Data.db') > ERROR [CompactionExecutor:5838] 2015-09-06 11:33:39,639 > CassandraDaemon.java:223 - Exception in thread > Thread[CompactionExecutor:5838,1,main] > java.lang.AssertionError: Memory was freed > at > org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:97) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at org.apache.cassandra.io.util.Memory.getInt(Memory.java:281) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:139) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:144) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:113) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:1183) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.firstKeyBeyond(SSTableReader.java:1667) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.needsCleanup(CompactionManager.java:693) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.doCleanupOne(CompactionManager.java:734) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:94) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:389) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:285) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_51] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > {code} > And: > {code} > ERROR [IndexSummaryManager:1] 2015-09-06 09:32:23,346 > CassandraDaemon.java:223 - Exception in thread > Thread[IndexSummaryManager:1,1,main] > java.lang.AssertionError: null > at > org.apache.cassandra.io.sstable.SSTableReader.setReplacedBy(SSTableReader.java:955) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneAndReplace(SSTableReader.java:1002) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneWithNewSummarySamplingLevel(SSTableReader.java:1105) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.adjustSamplingLevels(IndexSummaryManager.java:421) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:299) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139) > ~[apache-cassandra-2.1.7.jar:2
[jira] [Commented] (CASSANDRA-10274) Assertion Errors when interrupting Cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733924#comment-14733924 ] Marcus Eriksson commented on CASSANDRA-10274: - +1 > Assertion Errors when interrupting Cleanup > -- > > Key: CASSANDRA-10274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10274 > Project: Cassandra > Issue Type: Bug >Reporter: Jeff Jirsa >Assignee: Benedict >Priority: Critical > Fix For: 2.1.x > > > Exceptions encountered after Interrupting cleanup on 2.1.7 - logging due to > nature. Seen on 2 different nodes. May be related to CASSANDRA-10260 . > {code} > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,630 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 44698243387/139158 > 624314)bytes > INFO [CompactionExecutor:5838] 2015-09-06 11:33:39,638 > CompactionManager.java:1286 - Compaction interrupted: > Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, > 37886026781/133123 > 638379)bytes > INFO [CompactionExecutor:5836] 2015-09-06 11:33:39,639 > CompactionManager.java:749 - Cleaning up > SSTableReader(path='/mnt/cassandra/data/keyspace/table-74bffc100fbe11e5a5ce37a8b36fe285/keyspace-table-26598-Data.db') > ERROR [CompactionExecutor:5838] 2015-09-06 11:33:39,639 > CassandraDaemon.java:223 - Exception in thread > Thread[CompactionExecutor:5838,1,main] > java.lang.AssertionError: Memory was freed > at > org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:97) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at org.apache.cassandra.io.util.Memory.getInt(Memory.java:281) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:139) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:144) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:113) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:1183) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.firstKeyBeyond(SSTableReader.java:1667) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.needsCleanup(CompactionManager.java:693) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.doCleanupOne(CompactionManager.java:734) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:94) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:389) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:285) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > ~[na:1.8.0_51] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_51] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > {code} > And: > {code} > ERROR [IndexSummaryManager:1] 2015-09-06 09:32:23,346 > CassandraDaemon.java:223 - Exception in thread > Thread[IndexSummaryManager:1,1,main] > java.lang.AssertionError: null > at > org.apache.cassandra.io.sstable.SSTableReader.setReplacedBy(SSTableReader.java:955) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneAndReplace(SSTableReader.java:1002) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.SSTableReader.cloneWithNewSummarySamplingLevel(SSTableReader.java:1105) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.adjustSamplingLevels(IndexSummaryManager.java:421) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:299) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238) > ~[apache-cassandra-2.1.7.jar:2.1.7] > at > org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139) > ~[apache-cassandra-2.1
[jira] [Updated] (CASSANDRA-10280) Make DTCS work well with old data
[ https://issues.apache.org/jira/browse/CASSANDRA-10280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-10280: Description: Operational tasks become incredibly expensive if you keep around a long timespan of data with DTCS - with default settings and 1 year of data, the oldest window covers about 180 days. Bootstrapping a node with vnodes with this data layout will force cassandra to compact very many sstables in this window. We should probably put a cap on how big the biggest windows can get. We could probably default this to something sane based on max_sstable_age (ie, say we can reasonably handle 1000 sstables per node, then we can calculate how big the windows should be to allow that) was: Operational tasks become incredibly expensive if you keep around a long timespan of data with DTCS - with default settings and 1 year of data, the oldest window covers about 180 days. Bootstrapping a node with vnodes with this data layout will force cassandra to compact very many sstables in this window. We should probably put a cap on how big the biggest windows can get. We could probably default this to something sane based on max_sstable_age (ie, say we can reasonably handle 1000 sstables per node, then we can make calculate how big the windows should be to allow that) > Make DTCS work well with old data > - > > Key: CASSANDRA-10280 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10280 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.x, 2.1.x, 2.2.x > > > Operational tasks become incredibly expensive if you keep around a long > timespan of data with DTCS - with default settings and 1 year of data, the > oldest window covers about 180 days. Bootstrapping a node with vnodes with > this data layout will force cassandra to compact very many sstables in this > window. > We should probably put a cap on how big the biggest windows can get. We could > probably default this to something sane based on max_sstable_age (ie, say we > can reasonably handle 1000 sstables per node, then we can calculate how big > the windows should be to allow that) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8630) Faster sequential IO (on compaction, streaming, etc)
[ https://issues.apache.org/jira/browse/CASSANDRA-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733944#comment-14733944 ] Ariel Weisberg commented on CASSANDRA-8630: --- Since the work is already done I would say make the changes? FB.UR_UNINIT_READ_CALLED_FROM_SUPER_CONSTRUCTOR seems easy to justify and FB.NS_DANGEROUS_NON_SHORT_CIRCUIT is pretty low value, but simple enough that it's worth making coverity happy. > Faster sequential IO (on compaction, streaming, etc) > > > Key: CASSANDRA-8630 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8630 > Project: Cassandra > Issue Type: Improvement > Components: Core, Tools >Reporter: Oleg Anastasyev >Assignee: Stefania > Labels: compaction, performance > Fix For: 3.x > > Attachments: 8630-FasterSequencialReadsAndWrites.txt, cpu_load.png, > flight_recorder_001_files.tar.gz, flight_recorder_002_files.tar.gz, > mmaped_uncomp_hotspot.png > > > When node is doing a lot of sequencial IO (streaming, compacting, etc) a lot > of CPU is lost in calls to RAF's int read() and DataOutputStream's write(int). > This is because default implementations of readShort,readLong, etc as well as > their matching write* are implemented with numerous calls of byte by byte > read and write. > This makes a lot of syscalls as well. > A quick microbench shows than just reimplementation of these methods in > either way gives 8x speed increase. > A patch attached implements RandomAccessReader.read and > SequencialWriter.write methods in more efficient way. > I also eliminated some extra byte copies in CompositeType.split and > ColumnNameHelper.maxComponents, which were on my profiler's hotspot method > list during tests. > A stress tests on my laptop show that this patch makes compaction 25-30% > faster on uncompressed sstables and 15% faster for compressed ones. > A deployment to production shows much less CPU load for compaction. > (I attached a cpu load graph from one of our production, orange is niced CPU > load - i.e. compaction; yellow is user - i.e. not compaction related tasks) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10276) With DTCS, do STCS in windows if more than max_threshold sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734051#comment-14734051 ] Jeff Jirsa commented on CASSANDRA-10276: If your most recent window/bucket with compactable sstables has more than max_threshold sstables, but they're sized in such a way that stcs won't combine any of them, {{SizeTieredCompactionStrategy.mostInterestingBucket}} may return empty, which then causes {{trimToThreshold}} and {{newestBucket}} to return empty lists. If you may have older buckets that could otherwise be compacted in some way, they'll starve. If you check to see if {{SizeTieredCompactionStrategy.mostInterestingBucket(...).isEmpty()}} , you can continue to check older time windows/buckets/targets. This also impacts calculation of {{estimatedRemainingTasks}} Other than that it looks good. > With DTCS, do STCS in windows if more than max_threshold sstables > - > > Key: CASSANDRA-10276 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10276 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.x, 2.1.x, 2.2.x > > > To avoid constant recompaction of files in big ( > max threshold) DTCS > windows, we should do STCS of those files. > Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10109) Windows dtest 3.0: ttl_test.py failures
[ https://issues.apache.org/jira/browse/CASSANDRA-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734082#comment-14734082 ] Stefania commented on CASSANDRA-10109: -- [~benedict], CI seems OK so far; I've launched one more run just now, after rebasing. > Windows dtest 3.0: ttl_test.py failures > --- > > Key: CASSANDRA-10109 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10109 > Project: Cassandra > Issue Type: Sub-task >Reporter: Joshua McKenzie >Assignee: Stefania > Labels: Windows > Fix For: 3.0.0 rc1 > > > ttl_test.py:TestTTL.update_column_ttl_with_default_ttl_test2 > ttl_test.py:TestTTL.update_multiple_columns_ttl_test > ttl_test.py:TestTTL.update_single_column_ttl_test > Errors locally are different than CI from yesterday. Yesterday on CI we have > timeouts and general node hangs. Today on all 3 tests when run locally I see: > {noformat} > Traceback (most recent call last): > File "c:\src\cassandra-dtest\dtest.py", line 532, in tearDown > raise AssertionError('Unexpected error in %s node log: %s' % (node.name, > errors)) > AssertionError: Unexpected error in node1 node log: ['ERROR [main] 2015-08-17 > 16:53:43,120 NoSpamLogger.java:97 - This platform does not support atomic > directory streams (SecureDirectoryStream); race conditions when loading > sstable files could occurr'] > {noformat} > This traces back to the commit for CASSANDRA-7066 today by [~Stefania] and > [~benedict]. Stefania - care to take this ticket and also look further into > whether or not we're going to have issues with 7066 on Windows? That error > message certainly *sounds* like it's not a good thing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7392) Abort in-progress queries that time out
[ https://issues.apache.org/jira/browse/CASSANDRA-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734087#comment-14734087 ] Stefania commented on CASSANDRA-7392: - [~aweisberg] sorry for the long delay, I have finally found the time to get back to this: * Introduced a new package {{o.a.c.db.monitoring}} where all the monitoring classes are located. * Introduced a new interface, {{Monitorable}}, which is implemented by {{ReadCommand}} to ensure we get the CQL string only when needed, it also hosts all other required properties via {{MonitorableImpl}}: construction time, timeout and state. A {{Monitorable}} is basically an operation. * Renamed {{OpState}} to {{MonitoringStateRef}}, which extends {{AtomicReference}}; {{MonitoringState}} is just an enum. * Renamed {{OpMonitoring}} to {{MonitoringTask}}, which is a {{Runnable}} scheduled periodically to check a COW array of atomic references to operations. The atomic references are stored in thread local objects, {{MonitorableThreadLocal}} and their payload is updated by the clients that wish to have an operation monitored. * Implemented aggregated logging in {{MonitoringTask}}. * Extended utests. I believe this should address all of the comments discussed above except for: bq. In DroppableRunnable you switch from System.currentTimeMillis() to nanoTime(). I always looking for opportunities to avoid getting the current time (get it once and pass it around). We grab the time once when reading a message so theoretically if there has been no blocking you don't need to grab it again if we had ponies. Not sure if there is an opportunity for that here. There is no message involved here so I don't think we can avoid grabbing the time again. -- Please note new 3.0 branch: https://github.com/stef1927/cassandra/commits/7392-3.0 Pull request for updated dtests: https://github.com/riptano/cassandra-dtest/pull/535 CI: http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-7392-3.0-dtest/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-7392-3.0-testall/ > Abort in-progress queries that time out > --- > > Key: CASSANDRA-7392 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7392 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: Stefania >Priority: Critical > Fix For: 3.x > > > Currently we drop queries that time out before we get to them (because node > is overloaded) but not queries that time out while being processed. > (Particularly common for index queries on data that shouldn't be indexed.) > Adding the latter and logging when we have to interrupt one gets us a poor > man's "slow query log" for free. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10066) Bring cqlsh into PEP8 compliance
[ https://issues.apache.org/jira/browse/CASSANDRA-10066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734107#comment-14734107 ] Stefania commented on CASSANDRA-10066: -- dtests look good. Patch also looks good but I am not sure how to check with {{pep8}} so I simply tried the following command on the 2.1 patch and I got errors other than line too long (E501): {code} stefi@lila:~/git/cstar/cassandra/bin$ pep8 cqlsh | grep -v "E501" cqlsh:109:1: E402 module level import not at top of file cqlsh:110:1: E402 module level import not at top of file cqlsh:111:1: E402 module level import not at top of file cqlsh:112:1: E402 module level import not at top of file cqlsh:113:1: E402 module level import not at top of file cqlsh:114:1: E402 module level import not at top of file cqlsh:122:1: E402 module level import not at top of file cqlsh:123:1: E402 module level import not at top of file cqlsh:125:1: E402 module level import not at top of file cqlsh:127:1: E402 module level import not at top of file cqlsh:128:1: E402 module level import not at top of file cqlsh:1841:17: E731 do not assign a lambda expression, use a def {code} Are these errors expected or not? > Bring cqlsh into PEP8 compliance > > > Key: CASSANDRA-10066 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10066 > Project: Cassandra > Issue Type: Bug >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: cqlsh > Fix For: 3.x, 2.1.x, 2.2.x > > > In order for us to begin running flake8 against cqlsh.py in CI, it would be > helpful if it were already PEP8 complaint, with the exception of using 120 > character lines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10205) decommissioned_wiped_node_can_join_test fails on Jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734114#comment-14734114 ] Stefania commented on CASSANDRA-10205: -- [~philipthompson] could you set-up a Jenkins job for [this dtest patch|https://github.com/stef1927/cassandra-dtest/commits/10205] to run on [this C* patch|https://github.com/stef1927/cassandra/commits/10205-3.0] so I can continue testing? > decommissioned_wiped_node_can_join_test fails on Jenkins > > > Key: CASSANDRA-10205 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10205 > Project: Cassandra > Issue Type: Test >Reporter: Stefania >Assignee: Stefania > Attachments: decommissioned_wiped_node_can_join_test.tar.gz > > > This test passes locally but reliably fails on Jenkins. It seems after we > restart node4, it is unable to Gossip with other nodes: > {code} > INFO [HANDSHAKE-/127.0.0.2] 2015-08-27 06:50:42,778 > OutboundTcpConnection.java:494 - Handshaking version with /127.0.0.2 > INFO [HANDSHAKE-/127.0.0.1] 2015-08-27 06:50:42,778 > OutboundTcpConnection.java:494 - Handshaking version with /127.0.0.1 > INFO [HANDSHAKE-/127.0.0.3] 2015-08-27 06:50:42,778 > OutboundTcpConnection.java:494 - Handshaking version with /127.0.0.3 > ERROR [main] 2015-08-27 06:51:13,785 CassandraDaemon.java:635 - Exception > encountered during startup > java.lang.RuntimeException: Unable to gossip with any seeds > at > org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1342) > ~[main/:na] > at > org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:518) > ~[main/:na] > at > org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:763) > ~[main/:na] > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:687) > ~[main/:na] > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:570) > ~[main/:na] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:320) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:516) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:622) > [main/:na] > WARN [StorageServiceShutdownHook] 2015-08-27 06:51:13,799 Gossiper.java:1453 > - No local state or state is in silent shutdown, not announcing shutdown > {code} > It seems both the addresses and port number of the seeds are correct so I > don't think the problem is the Amazon private addresses but I might be wrong. > It's also worth noting that the first time the node starts up without > problems. The problem only occurs during a restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10255) AttributeError: 'module' object has no attribute 'compress'
[ https://issues.apache.org/jira/browse/CASSANDRA-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] navithejesh resolved CASSANDRA-10255. - Resolution: Fixed > AttributeError: 'module' object has no attribute 'compress' > > > Key: CASSANDRA-10255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10255 > Project: Cassandra > Issue Type: Bug > Components: Config, Packaging > Environment: Linux fedora 21 >Reporter: navithejesh > Labels: cqlsh > Fix For: 2.1.x > > Attachments: erroe2.jpeg, error.jpeg > > > Hi Recently i have upgrade my server from cassandra version 2.0.11 to 2.1.8 > using tarball installation.And i have added one node to existing culster with > the same version of 2.1.8. All the configuration settings are done but am not > able to run cqlsh from bin directory.and this is the error > AttributeError: 'module' object has no attribute 'compress' > {code} > ./cqlsh > Traceback (most recent call last): > File "./cqlsh", line 111, in > from cassandra.cluster import Cluster, PagedResult > File > "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/cluster.py", > line 49, in > File > "/home/cassandra/apache-cassandra-2.1.8/bin/../lib/cassandra-driver-internal-only-2.6.0c2.post.zip/cassandra-driver-2.6.0c2.post/cassandra/connection.py", > line 88, in > AttributeError: 'module' object has no attribute 'compress' > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal
[ https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-10089: - Fix Version/s: 3.0.0 rc1 2.2.x 2.1.x > NullPointerException in Gossip handleStateNormal > > > Key: CASSANDRA-10089 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10089 > Project: Cassandra > Issue Type: Bug >Reporter: Stefania >Assignee: Stefania > Fix For: 2.1.x, 2.2.x, 3.0.x > > > Whilst comparing dtests for CASSANDRA-9970 I found [this failing > dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/] > in 2.2: > {code} > Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 > 15:39:57,873 CassandraDaemon.java:183 - Exception in thread > Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat > org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) > ~[main/:na] \tat > org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49) > ~[main/:na] \tat > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) > ~[main/:na] \tat > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_80] \tat > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]'] > {code} > I wasn't able to find it on unpatched branches but it is clearly not related > to CASSANDRA-9970, if anything it could have been a side effect of > CASSANDRA-9871. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal
[ https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-10089: - Fix Version/s: (was: 3.0.0 rc1) 3.0.x > NullPointerException in Gossip handleStateNormal > > > Key: CASSANDRA-10089 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10089 > Project: Cassandra > Issue Type: Bug >Reporter: Stefania >Assignee: Stefania > Fix For: 2.1.x, 2.2.x, 3.0.x > > > Whilst comparing dtests for CASSANDRA-9970 I found [this failing > dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/] > in 2.2: > {code} > Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 > 15:39:57,873 CassandraDaemon.java:183 - Exception in thread > Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat > org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) > ~[main/:na] \tat > org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49) > ~[main/:na] \tat > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) > ~[main/:na] \tat > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_80] \tat > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]'] > {code} > I wasn't able to find it on unpatched branches but it is clearly not related > to CASSANDRA-9970, if anything it could have been a side effect of > CASSANDRA-9871. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal
[ https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734214#comment-14734214 ] Stefania commented on CASSANDRA-10089: -- The NPE can be seen also in 2.1 [here|http://cassci.datastax.com/job/cassandra-2.1_dtest/241/testReport/consistency_test/TestConsistency/short_read_test/] so I will prepare a patch for 2.1+. > NullPointerException in Gossip handleStateNormal > > > Key: CASSANDRA-10089 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10089 > Project: Cassandra > Issue Type: Bug >Reporter: Stefania >Assignee: Stefania > Fix For: 2.1.x, 2.2.x, 3.0.x > > > Whilst comparing dtests for CASSANDRA-9970 I found [this failing > dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/] > in 2.2: > {code} > Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 > 15:39:57,873 CassandraDaemon.java:183 - Exception in thread > Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat > org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) > ~[main/:na] \tat > org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49) > ~[main/:na] \tat > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) > ~[main/:na] \tat > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_80] \tat > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]'] > {code} > I wasn't able to find it on unpatched branches but it is clearly not related > to CASSANDRA-9970, if anything it could have been a side effect of > CASSANDRA-9871. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10276) With DTCS, do STCS in windows if more than max_threshold sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-10276: Reviewer: Jeff Jirsa > With DTCS, do STCS in windows if more than max_threshold sstables > - > > Key: CASSANDRA-10276 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10276 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.x, 2.1.x, 2.2.x > > > To avoid constant recompaction of files in big ( > max threshold) DTCS > windows, we should do STCS of those files. > Patch here: https://github.com/krummas/cassandra/commits/marcuse/dtcs_stcs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10089) NullPointerException in Gossip handleStateNormal
[ https://issues.apache.org/jira/browse/CASSANDRA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734288#comment-14734288 ] Stefania commented on CASSANDRA-10089: -- I've attached a 2.1+ patch that prevents the NPE. I could not reproduce the problem locally despite running the offending dtests several times but I plan to run CI several times. The NPE should be definitely fixed, the question is whether having a node jumping to state normal with no tokens could cause issues. h3. Patches: https://github.com/stef1927/cassandra/commits/10089-2.1 https://github.com/stef1927/cassandra/commits/10089-2.2 https://github.com/stef1927/cassandra/commits/10089-3.0 h3. CI: http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10089-2.1-testall/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10089-2.1-dtest/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10089-2.2-testall/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10089-2.2-dtest/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10089-3.0-testall/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10089-3.0-dtest/ > NullPointerException in Gossip handleStateNormal > > > Key: CASSANDRA-10089 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10089 > Project: Cassandra > Issue Type: Bug >Reporter: Stefania >Assignee: Stefania > Fix For: 2.1.x, 2.2.x, 3.0.x > > > Whilst comparing dtests for CASSANDRA-9970 I found [this failing > dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-9970-dtest/lastCompletedBuild/testReport/consistency_test/TestConsistency/short_read_test/] > in 2.2: > {code} > Unexpected error in node1 node log: ['ERROR [GossipStage:1] 2015-08-14 > 15:39:57,873 CassandraDaemon.java:183 - Exception in thread > Thread[GossipStage:1,5,main] java.lang.NullPointerException: null \tat > org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1731) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1804) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1857) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onChange(StorageService.java:1629) > ~[main/:na] \tat > org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2312) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1025) > ~[main/:na] \tat > org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1106) > ~[main/:na] \tat > org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49) > ~[main/:na] \tat > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:66) > ~[main/:na] \tat > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_80] \tat > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_80] \tat java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]'] > {code} > I wasn't able to find it on unpatched branches but it is clearly not related > to CASSANDRA-9970, if anything it could have been a side effect of > CASSANDRA-9871. -- This message was sent by Atlassian JIRA (v6.3.4#6332)