[jira] [Commented] (CASSANDRA-7306) Support "edge dcs" with more flexible gossip
[ https://issues.apache.org/jira/browse/CASSANDRA-7306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023873#comment-15023873 ] Jeff Jirsa commented on CASSANDRA-7306: --- As I mentioned, I've implemented a good chunk of this. I believe it's a viable approach, but I'd love feedback: https://github.com/jeffjirsa/cassandra/tree/cassandra-7306 . This branch allows edge dcs as described, and extends NTS so that it can be updated of DC topology changes (and excludes ungossipable DCs from the mutation/read path, which may be a bit too much rope - maybe this needs to be dialed back). > Support "edge dcs" with more flexible gossip > > > Key: CASSANDRA-7306 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7306 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper > Labels: ponies > > As Cassandra clusters get bigger and bigger, and their topology becomes more > complex, there is more and more need for a notion of "hub" and "spoke" > datacenters. > One of the big obstacles to supporting hundreds (or thousands) of remote dcs, > is the assumption that all dcs need to talk to each other (and be connected > all the time). > This ticket is a vague placeholder with the goals of achieving: > 1) better behavioral support for occasionally disconnected datacenters > 2) explicit support for custom dc to dc routing. A simple approach would be > an optional per-dc annotation of which other DCs that DC could gossip with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9304) COPY TO improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023723#comment-15023723 ] Stefania commented on CASSANDRA-9304: - This should be fixed by this [pull request|https://github.com/riptano/cassandra-dtest/pull/682]. > COPY TO improvements > > > Key: CASSANDRA-9304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9304 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Stefania >Priority: Minor > Labels: cqlsh > Fix For: 2.1.12, 2.2.4, 3.0.1, 3.1 > > > COPY FROM has gotten a lot of love. COPY TO not so much. One obvious > improvement could be to parallelize reading and writing (write one page of > data while fetching the next). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10751) "Pool is shutdown" error when running Hadoop jobs on Yarn
[
https://issues.apache.org/jira/browse/CASSANDRA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023612#comment-15023612
]
Cyril Scetbon commented on CASSANDRA-10751:
---
Cassandra 2.1.11. It was working fine with 2.0.12
> "Pool is shutdown" error when running Hadoop jobs on Yarn
> -
>
> Key: CASSANDRA-10751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10751
> Project: Cassandra
> Issue Type: Bug
> Environment: Hadoop 2.7.1 (HDP 2.3.2)
> Cassandra 2.1.11
>Reporter: Cyril Scetbon
>Assignee: Alex Liu
> Attachments: output.log
>
>
> Trying to execute an Hadoop job on Yarn, I get errors from Cassandra's
> internal code. It seems that connections are shutdown but we can't understand
> why ...
> Here is a subtract of the errors. I also add a file with the complete debug
> logs.
> {code}
> 15/11/22 20:05:54 [main]: DEBUG core.RequestHandler: Error querying
> node006.internal.net/192.168.12.22:9042, trying next host (error is:
> com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown)
> Failed with exception java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> 15/11/22 20:05:54 [main]: ERROR CliDriver: Failed with exception
> java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> java.io.IOException: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.cassandra.input.cql.HiveCqlInputFormat.getRecordReader(HiveCqlInputFormat.java:132)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:674)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:324)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
> ... 15 more
> Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All
> host(s) tried for query failed (tried:
> node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
> at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> at
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:214)
> at
> com.datastax.driver.core.AbstractSession.execu
[jira] [Created] (CASSANDRA-10766) OOM from 22000 memtables
amorton created CASSANDRA-10766:
---
Summary: OOM from 22000 memtables
Key: CASSANDRA-10766
URL: https://issues.apache.org/jira/browse/CASSANDRA-10766
Project: Cassandra
Issue Type: Bug
Environment: * 32 cores, physical machines
* 8 SSD drives in raid 0
* hotspot java 8.60
* Linux version 3.8.13-98.el6uek.x86_64 (mockbuild@x86-ol6-builder-04) (gcc
version 4.4.7 20120313 (Red Hat 4.4.7-11)
Reporter: amorton
Attachments: MemtableFlushWriter.txt, SlabPoolCleaner.txt,
lots-of-memtables.png, thread-dump-flush.txt, thread-dump-full.txt
Hi folks, we observed this on a new cassandra 2.1.11 cluster.
The node ran out of memory, and when it did it had 22,000 + memtables in memory
with a corresponding number of pending MemtableFlushWriter and
MemtablePostFlushWriter tasks. We have seen other nodes going OOM, but have not
looked at them as deeply as this one.
We also noticed that while there was 8 flush writers configured, there were
times when only one (or maybe two) were making progress. And that there was
times when the SlabPoolCleaner was flushing memtables 50 times a second.
I'll try to put as much info below that makes sense. I have the full system
logs and a heap dump and can go through them with someone if needed. Please
ping me on IRC or the usual email if I am not on.
Below is part of the Heap Dump showing the huge amount of memtables in ram.
I have also attached copied of the same thread dump:
* thread-dump-full.txt is the full dump, node that is includes approximately
300 shared worker threads.
* thread-dump-flush.txt is the MemtableFlushWriter and MemtablePostFlush
threads.
Note that all the MemtableFlushWriter threads are trying to update the
DataTracker, and that the MemtablePostFlush is waiting on the countdown latch
to be set. The PostFlush object in the MemtablePostFlush thread is associated
with the MemtableFlushWriter:744 thread.
Looking at the MemtableFlushWriter threads they are trying to flush the
following tables:
{noformat}
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:750" 266120 120
system.hints
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:744" 223864 120
system.sstable_activity
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:752" 220848 120
system.sstable_activity
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:754" 219736 120
system.sstable_activity
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:749" 214336 120
system.sstable_activity
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:753" 211632 120
system.sstable_activity
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:748" 149616 120
system.sstable_activity
java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:751" 143480 120
system.sstable_activity
{noformat}
When I saw that these were all on the same table I was wondering if the CAS in
{{DataTracker.replaceFlushed()}} was starving some threads.
The attached SlabPoolCleaner.txt file is the lines from the SlabPoolCleaner
thread from a couple of system.log files. Notes:
* The machine went OOM at 08:38:25,463 server time.
* The server was flushing user tables between 100 and 200 MB until approx
2015-11-22 07:40:39,669 server time.
* After that the server was flushing files that were 10's of bytes.
The attached MemtableFlushWriter.txt includes log lines from the
MemtableFlushWriter threads. Notes:
* from approx 2015-11-22 07:41:18 it often looks like only one
MemtableFlushWriter thread progresses at a time.
A few other notes:
* This machine has logged a few dozen LEAK DETECTED messages.
* This has logged messages about corrupted SSTables.
At this point I am unsure if the SlabPoolCleaner was reacting to low memory and
correctly flushing, or it was incorrectly flushing a lot which overloaded the
flush system. We are also unsure what role the CAS in
{{DataTracker.replaceFlushed()}} played, all the MemtableFlushWriter threads
with the same progress seems odd.
As a test for the CAS theory we ran the node with {{memtable_flush_writers}}
set to 2, it has been running for approx 12 hours under load and has not OOM'd
yet. This is a good sign.
Any thoughts on what could be happening ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10766) OOM from 22000 memtables
[
https://issues.apache.org/jira/browse/CASSANDRA-10766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amorton updated CASSANDRA-10766:
Environment:
* 32 cores, physical machines
* 8 SSD drives in raid 0
* hotspot java 8.60
* Linux version 3.8.13-98.el6uek.x86_64 (mockbuild@x86-ol6-builder-04) (gcc
version 4.4.7 20120313 (Red Hat 4.4.7-11)
* G1 heap with 26 GB heap, 24 concurrent and 24 parallel threads, 500 ms pause
target
was:
* 32 cores, physical machines
* 8 SSD drives in raid 0
* hotspot java 8.60
* Linux version 3.8.13-98.el6uek.x86_64 (mockbuild@x86-ol6-builder-04) (gcc
version 4.4.7 20120313 (Red Hat 4.4.7-11)
> OOM from 22000 memtables
> -
>
> Key: CASSANDRA-10766
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10766
> Project: Cassandra
> Issue Type: Bug
> Environment: * 32 cores, physical machines
> * 8 SSD drives in raid 0
> * hotspot java 8.60
> * Linux version 3.8.13-98.el6uek.x86_64 (mockbuild@x86-ol6-builder-04) (gcc
> version 4.4.7 20120313 (Red Hat 4.4.7-11)
> * G1 heap with 26 GB heap, 24 concurrent and 24 parallel threads, 500 ms
> pause target
>Reporter: amorton
> Attachments: MemtableFlushWriter.txt, SlabPoolCleaner.txt,
> lots-of-memtables.png, thread-dump-flush.txt, thread-dump-full.txt
>
>
> Hi folks, we observed this on a new cassandra 2.1.11 cluster.
> The node ran out of memory, and when it did it had 22,000 + memtables in
> memory with a corresponding number of pending MemtableFlushWriter and
> MemtablePostFlushWriter tasks. We have seen other nodes going OOM, but have
> not looked at them as deeply as this one.
> We also noticed that while there was 8 flush writers configured, there were
> times when only one (or maybe two) were making progress. And that there was
> times when the SlabPoolCleaner was flushing memtables 50 times a second.
> I'll try to put as much info below that makes sense. I have the full system
> logs and a heap dump and can go through them with someone if needed. Please
> ping me on IRC or the usual email if I am not on.
> Below is part of the Heap Dump showing the huge amount of memtables in ram.
> I have also attached copied of the same thread dump:
> * thread-dump-full.txt is the full dump, node that is includes approximately
> 300 shared worker threads.
> * thread-dump-flush.txt is the MemtableFlushWriter and MemtablePostFlush
> threads.
> Note that all the MemtableFlushWriter threads are trying to update the
> DataTracker, and that the MemtablePostFlush is waiting on the countdown latch
> to be set. The PostFlush object in the MemtablePostFlush thread is associated
> with the MemtableFlushWriter:744 thread.
> Looking at the MemtableFlushWriter threads they are trying to flush the
> following tables:
> {noformat}
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:750" 266120 120
> system.hints
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:744" 223864 120
> system.sstable_activity
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:752" 220848 120
> system.sstable_activity
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:754" 219736 120
> system.sstable_activity
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:749" 214336 120
> system.sstable_activity
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:753" 211632 120
> system.sstable_activity
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:748" 149616 120
> system.sstable_activity
> java.lang.Thread [Thread, Stack Local] "MemtableFlushWriter:751" 143480 120
> system.sstable_activity
> {noformat}
> When I saw that these were all on the same table I was wondering if the CAS
> in {{DataTracker.replaceFlushed()}} was starving some threads.
> The attached SlabPoolCleaner.txt file is the lines from the SlabPoolCleaner
> thread from a couple of system.log files. Notes:
> * The machine went OOM at 08:38:25,463 server time.
> * The server was flushing user tables between 100 and 200 MB until approx
> 2015-11-22 07:40:39,669 server time.
> * After that the server was flushing files that were 10's of bytes.
> The attached MemtableFlushWriter.txt includes log lines from the
> MemtableFlushWriter threads. Notes:
> * from approx 2015-11-22 07:41:18 it often looks like only one
> MemtableFlushWriter thread progresses at a time.
> A few other notes:
> * This machine has logged a few dozen LEAK DETECTED messages.
> * This has logged messages about corrupted SSTables.
> At this point I am unsure if the SlabPoolCleaner was reacting to low memory
> and correctly flushing, or it was incorrectly flushing a lot which overloaded
> the flush system. We are also unsure what role the CAS in
> {{DataTracker.replaceFlushed()}} played, all the MemtableFlushWriter threads
> with the same progress seems odd.
> As a test fo
[jira] [Comment Edited] (CASSANDRA-10243) Warn or fail when changing cluster topology live
[ https://issues.apache.org/jira/browse/CASSANDRA-10243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023553#comment-15023553 ] Stefania edited comment on CASSANDRA-10243 at 11/24/15 1:31 AM: Thanks for your review. Here is the next update: bq. It's not very common to update snitch property of a live node (specially with GPFS), so I don't think the warning is necessary. Should we maybe add a note to NEWS.txt instead? Or maybe print the warn on 2.1/2.2, and remove it on 3.0? Added a note to NEWS.txt and removed the warning. bq. On CASSANDRA-9474 we changed the property name from override_rackdc to the original ignore_rack in addition to ignore_dc, could you please fix dtests? I've updated the tests and removed the @require on 9474 (I've added ignore_rack to 2.2 so we can run the full tests on Jenkins via DTEST_REPO and DTEST_BRANCH). bq. Instead of manipulating gossip EndpointState in StorageProxy.truncateBlocking(), what do you think about adding a excludeNodesInDeadState option to StorageService.getLiveMembers and Gossiper.getLiveMembers (which you could also rename it to Gossiper.getLiveEndpoints)? Done bq. Please fix nodetool_test.TestNodetool.test_correct_dc_rack_in_nodetool_info dtest Now that the GPFS error is gone this test is fine, thanks for checking this. bq. It seems 3.1 dtest did not run, mind resubmitting please? Submitted together with the other branches just now. The reason I did not submit it yesterday is because the 3.1 branch is identical to the 3.0 branch (checked with git diff). I suppose I should have not added to the table above either! :) I'll post another update once CI completes. was (Author: stefania): Thanks for your review. Here is the next update: bq. It's not very common to update snitch property of a live node (specially with GPFS), so I don't think the warning is necessary. Should we maybe add a note to NEWS.txt instead? Or maybe print the warn on 2.1/2.2, and remove it on 3.0? Added a note to NEWS.txt and removed the warning. bq. On CASSANDRA-9474 we changed the property name from override_rackdc to the original ignore_rack in addition to ignore_dc, could you please fix dtests? I've updated the tests and removed the @require on 9474 (I've added ignore_rack to 2.2 so we can run the full tests on Jenkins via DTEST_REPO and DTEST_BRANCH). bq. Instead of manipulating gossip EndpointState in StorageProxy.truncateBlocking(), what do you think about adding a excludeNodesInDeadState option to StorageService.getLiveMembers and Gossiper.getLiveMembers (which you could also rename it to Gossiper.getLiveEndpoints)? Done bq. Please fix nodetool_test.TestNodetool.test_correct_dc_rack_in_nodetool_info dtest Now that the GPFS error is gone this test is fine, thanks for checking this. bq. It seems 3.1 dtest did not run, mind resubmitting please? Submitted together with the other branches just now. The reason I did not submit it yesterday is because the 3.1 branch is identical to the 3.0 branch (checked with git diff). I'll post another update once CI completes. > Warn or fail when changing cluster topology live > > > Key: CASSANDRA-10243 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10243 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Jonathan Ellis >Assignee: Stefania >Priority: Critical > Fix For: 2.1.x > > > Moving a node from one rack to another in the snitch, while it is alive, is > almost always the wrong thing to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10243) Warn or fail when changing cluster topology live
[ https://issues.apache.org/jira/browse/CASSANDRA-10243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023553#comment-15023553 ] Stefania commented on CASSANDRA-10243: -- Thanks for your review. Here is the next update: bq. It's not very common to update snitch property of a live node (specially with GPFS), so I don't think the warning is necessary. Should we maybe add a note to NEWS.txt instead? Or maybe print the warn on 2.1/2.2, and remove it on 3.0? Added a note to NEWS.txt and removed the warning. bq. On CASSANDRA-9474 we changed the property name from override_rackdc to the original ignore_rack in addition to ignore_dc, could you please fix dtests? I've updated the tests and removed the @require on 9474 (I've added ignore_rack to 2.2 so we can run the full tests on Jenkins via DTEST_REPO and DTEST_BRANCH). bq. Instead of manipulating gossip EndpointState in StorageProxy.truncateBlocking(), what do you think about adding a excludeNodesInDeadState option to StorageService.getLiveMembers and Gossiper.getLiveMembers (which you could also rename it to Gossiper.getLiveEndpoints)? Done bq. Please fix nodetool_test.TestNodetool.test_correct_dc_rack_in_nodetool_info dtest Now that the GPFS error is gone this test is fine, thanks for checking this. bq. It seems 3.1 dtest did not run, mind resubmitting please? Submitted together with the other branches just now. The reason I did not submit it yesterday is because the 3.1 branch is identical to the 3.0 branch (checked with git diff). I'll post another update once CI completes. > Warn or fail when changing cluster topology live > > > Key: CASSANDRA-10243 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10243 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Jonathan Ellis >Assignee: Stefania >Priority: Critical > Fix For: 2.1.x > > > Moving a node from one rack to another in the snitch, while it is alive, is > almost always the wrong thing to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10751) "Pool is shutdown" error when running Hadoop jobs on Yarn
[
https://issues.apache.org/jira/browse/CASSANDRA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023480#comment-15023480
]
Alex Liu commented on CASSANDRA-10751:
--
The error indicates that C* node can't handle the load(Maybe there are too many
splits). Do you try the latest C* 2.x or C*3.x?
> "Pool is shutdown" error when running Hadoop jobs on Yarn
> -
>
> Key: CASSANDRA-10751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10751
> Project: Cassandra
> Issue Type: Bug
> Environment: Hadoop 2.7.1 (HDP 2.3.2)
> Cassandra 2.1.11
>Reporter: Cyril Scetbon
>Assignee: Alex Liu
> Attachments: output.log
>
>
> Trying to execute an Hadoop job on Yarn, I get errors from Cassandra's
> internal code. It seems that connections are shutdown but we can't understand
> why ...
> Here is a subtract of the errors. I also add a file with the complete debug
> logs.
> {code}
> 15/11/22 20:05:54 [main]: DEBUG core.RequestHandler: Error querying
> node006.internal.net/192.168.12.22:9042, trying next host (error is:
> com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown)
> Failed with exception java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> 15/11/22 20:05:54 [main]: ERROR CliDriver: Failed with exception
> java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> java.io.IOException: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.cassandra.input.cql.HiveCqlInputFormat.getRecordReader(HiveCqlInputFormat.java:132)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:674)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:324)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
> ... 15 more
> Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All
> host(s) tried for query failed (tried:
> node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
> at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> at
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.j
[jira] [Commented] (CASSANDRA-10541) cqlshlib tests cannot run on Windows
[
https://issues.apache.org/jira/browse/CASSANDRA-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023476#comment-15023476
]
Paulo Motta commented on CASSANDRA-10541:
-
Windows test results are a bit strange. There was [one
success|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-3.0-paulomotta/12/testReport/]
in the 3.0 branch, and different failures on different test runs:
* [build/checkout
failures|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-trunk-paulomotta/2/console]
* [control connection
errors|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-2.2-paulomotta/2/]
* [assertion
errors|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-2.2-paulomotta/1/testReport/junit/cqlshlib.test.test_cqlsh_output/TestCqlshOutput/test_describe_columnfamily_output/]
* [type parsing
errors|http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cqlshlib_tests-windows-2.2-paulomotta/1/testReport/junit/cqlshlib.test.test_cqlsh_output/TestCqlshOutput/test_describe_schema_output/]
I'm not able to reproduce any of these errors locally. I wonder if there's
something strange with the environment? Is the environment setup in the same
way as on Linux? What python version is it running? How is the cassandra test
instance instantiated? Any thoughts [~mambocab] ?
> cqlshlib tests cannot run on Windows
>
>
> Key: CASSANDRA-10541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10541
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Benjamin Lerer
>Assignee: Paulo Motta
>Priority: Minor
> Labels: cqlsh, windows
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> If I try to run the {{cqlshlib}} tests on Windows, I got the following error:
> {quote}
> ==
> ERROR: Failure: AttributeError ('module' object has no attribute 'symlink')
> --
> Traceback (most recent call last):
> File "C:\Python27\lib\site-packages\nose\loader.py", line 414, in
> loadTestsFromName
> addr.filename, addr.module)
> File "C:\Python27\lib\site-packages\nose\importer.py", line 47, in
> importFromPath
> return self.importFromDir(dir_path, fqname)
> File "C:\Python27\lib\site-packages\nose\importer.py", line 94, in
> importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
> File "[...]\pylib\cqlshlib\test\__init__.py", line 17, in
> from .cassconnect import create_test_db, remove_test_db
> File "[...]\pylib\cqlshlib\test\cassconnect.py", line 22, in
> from .basecase import cql, cqlsh, cqlshlog, TEST_HOST, TEST_PORT, rundir
> File "[...]\pylib\cqlshlib\test\basecase.py", line 43, in
> os.symlink(path_to_cqlsh, modulepath)
> AttributeError: 'module' object has no attribute 'symlink'
> --
> Ran 1 test in 0.002s
> FAILED (errors=1)
> {quote}
> The problem comes from the fact tha Windows has no support for symlinks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-10738) upgrading to v3.0.0 nodes crashes with "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous bytes after map value"
[ https://issues.apache.org/jira/browse/CASSANDRA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cs reopened CASSANDRA-10738: > upgrading to v3.0.0 nodes crashes with > "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value" > --- > > Key: CASSANDRA-10738 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10738 > Project: Cassandra > Issue Type: Bug > Environment: ubuntu14, java8 >Reporter: cs >Assignee: Aleksey Yeschenko > Attachments: cassv3.txt > > > upgrading from v.2.2.3 to v3.0.0 > restarting after upgrade crashes with either > ERROR [main] 2015-11-19 20:21:26,511 CassandraDaemon.java:702 - Exception > encountered during startup > org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value > OR > ERROR [main] 2015-11-19 20:01:13,945 CassandraDaemon.java:702 - Exception > encountered during startup > java.lang.IllegalArgumentException: null > see attached debug and schema -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10738) upgrading to v3.0.0 nodes crashes with "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous bytes after map value"
[ https://issues.apache.org/jira/browse/CASSANDRA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023461#comment-15023461 ] cs commented on CASSANDRA-10738: additionally, using the snapshots from the data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/snapshots/ still results in: DEBUG [SSTableBatchOpen:1] 2015-11-24 00:16:40,298 SSTableReader.java:470 - Opening /var/lib/cassandra/data/system_schema/indexes-0feb57ac311f382fba6d9024d305702f/ma-1-big (51 bytes) ERROR [main] 2015-11-24 00:16:40,311 CassandraDaemon.java:702 - Exception encountered during startup java.lang.IllegalArgumentException: null at java.nio.Buffer.limit(Buffer.java:275) ~[na:1.8.0_45] at org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:611) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.utils.ByteBufferUtil.readBytesWithShortLength(ByteBufferUtil.java:620) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.marshal.CompositeType.extractComponent(CompositeType.java:231) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.LegacyLayout.decodeCellName(LegacyLayout.java:142) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.LegacyLayout.readLegacyCellBody(LegacyLayout.java:1002) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.LegacyLayout.readLegacyAtom(LegacyLayout.java:956) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$AtomIterator.readAtom(UnfilteredDeserializer.java:520) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$AtomIterator.hasNext(UnfilteredDeserializer.java:503) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:412) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:289) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.readStaticRow(AbstractSSTableIterator.java:147) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.(AbstractSSTableIterator.java:103) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.columniterator.SSTableIterator.(SSTableIterator.java:46) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:69) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:333) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:65) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:66) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:62) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:610) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:371) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:387) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:76) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:294) ~[apache-cassandra-3.0.0.jar:3.0.0] at org.apache.cassandra.schema.SchemaKeyspace.query(SchemaKeyspace.java:1238) ~[apache-cassandra-3.0.0.jar:3.0.0] > upgrading to v3.0.0 nodes crashes with > "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value" > -
[jira] [Commented] (CASSANDRA-10751) "Pool is shutdown" error when running Hadoop jobs on Yarn
[
https://issues.apache.org/jira/browse/CASSANDRA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023401#comment-15023401
]
Cyril Scetbon commented on CASSANDRA-10751:
---
Hey [~alexliu68], I've assigned it to you as I know you're the one who can
easily guess what happens. Don't hesitate to assign it to someone else if you
think I'm wrong. Thanks
> "Pool is shutdown" error when running Hadoop jobs on Yarn
> -
>
> Key: CASSANDRA-10751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10751
> Project: Cassandra
> Issue Type: Bug
> Environment: Hadoop 2.7.1 (HDP 2.3.2)
> Cassandra 2.1.11
>Reporter: Cyril Scetbon
>Assignee: Alex Liu
> Attachments: output.log
>
>
> Trying to execute an Hadoop job on Yarn, I get errors from Cassandra's
> internal code. It seems that connections are shutdown but we can't understand
> why ...
> Here is a subtract of the errors. I also add a file with the complete debug
> logs.
> {code}
> 15/11/22 20:05:54 [main]: DEBUG core.RequestHandler: Error querying
> node006.internal.net/192.168.12.22:9042, trying next host (error is:
> com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown)
> Failed with exception java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> 15/11/22 20:05:54 [main]: ERROR CliDriver: Failed with exception
> java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> java.io.IOException: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.cassandra.input.cql.HiveCqlInputFormat.getRecordReader(HiveCqlInputFormat.java:132)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:674)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:324)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
> ... 15 more
> Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All
> host(s) tried for query failed (tried:
> node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
> at
> com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
> at
> com.datastax.driver.core.Defaul
[jira] [Comment Edited] (CASSANDRA-10754) Failure in QueryPagerTest.multiQueryTest due to assertion in SinglePartitionReadCommand$Group
[
https://issues.apache.org/jira/browse/CASSANDRA-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023380#comment-15023380
]
Joel Knighton edited comment on CASSANDRA-10754 at 11/23/15 11:49 PM:
--
I've pushed a patch to 3.0
[here|https://github.com/jkni/cassandra/commits/10754-3.0].
I've changed all explicit references to {{nowInSeconds}} to a value generated
once during static initialization of the class.
I haven't run CI since this only affects one unit test; I induced the original
flaky failure by adding a sleep in command construction and confirmed that the
above patch solves the issues. All other tests in the class still pass.
was (Author: jkni):
I've pushed a patch to 3.0
[here|https://github.com/jkni/cassandra/commits/10754-3.0].
I've changed all explicit references to {{nowInSeconds}} to a value generated
once during static initialization of the class.
I haven't run CI since this only affects unit tests; I induced the original
flaky failure by adding a sleep in command construction and confirmed that the
above patch solves the issues. All other tests in the class still pass.
> Failure in QueryPagerTest.multiQueryTest due to assertion in
> SinglePartitionReadCommand$Group
> -
>
> Key: CASSANDRA-10754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10754
> Project: Cassandra
> Issue Type: Bug
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>
> Test failure
> [here|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-9510-3.0-testall/lastCompletedBuild/testReport/org.apache.cassandra.service/QueryPagerTest/multiQueryTest/].
> It's on a 3.0 branch for a issue, but the patch only affects the
> implementation of the {{assassinate}} command and wouldn't affect this unit
> test.
> The assertion failure is in the construction of a Group of
> SinglePartitionReadCommands, where the value of {{nowInSec}} for each command
> does not match.
> I'm not sure if 1) this invariant isn't valuable, 2) this invariant is
> valuable but the unit test does not accurately reflect how a group would be
> created, or 3) this is a genuine bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10754) Failure in QueryPagerTest.multiQueryTest due to assertion in SinglePartitionReadCommand$Group
[
https://issues.apache.org/jira/browse/CASSANDRA-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023380#comment-15023380
]
Joel Knighton commented on CASSANDRA-10754:
---
I've pushed a patch to 3.0
[here|https://github.com/jkni/cassandra/commits/10754-3.0].
I've changed all explicit references to {{nowInSeconds}} to a value generated
once during static initialization of the class.
I haven't run CI since this only affects unit tests; I induced the original
flaky failure by adding a sleep in command construction and confirmed that the
above patch solves the issues. All other tests in the class still pass.
> Failure in QueryPagerTest.multiQueryTest due to assertion in
> SinglePartitionReadCommand$Group
> -
>
> Key: CASSANDRA-10754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10754
> Project: Cassandra
> Issue Type: Bug
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>
> Test failure
> [here|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-9510-3.0-testall/lastCompletedBuild/testReport/org.apache.cassandra.service/QueryPagerTest/multiQueryTest/].
> It's on a 3.0 branch for a issue, but the patch only affects the
> implementation of the {{assassinate}} command and wouldn't affect this unit
> test.
> The assertion failure is in the construction of a Group of
> SinglePartitionReadCommands, where the value of {{nowInSec}} for each command
> does not match.
> I'm not sure if 1) this invariant isn't valuable, 2) this invariant is
> valuable but the unit test does not accurately reflect how a group would be
> created, or 3) this is a genuine bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10764) Cassandra Daemon should print JVM arguments
[ https://issues.apache.org/jira/browse/CASSANDRA-10764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Kale updated CASSANDRA-10764: - Description: While debugging, its useful to check if Cassandra indeed is picking up the JVM args supplied to it. Therefore, we should print those on startup. Patch is attached, and straightforward. was: While debugging, its useful to check if Cassandra indeed is picking up the JVM args supplied to it. Therefore, we should print those on startup. I'll submit a patch shortly. > Cassandra Daemon should print JVM arguments > --- > > Key: CASSANDRA-10764 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10764 > Project: Cassandra > Issue Type: Improvement > Components: Tools > Environment: PROD >Reporter: Anubhav Kale >Priority: Minor > Fix For: 2.1.12 > > Attachments: 10764.patch > > > While debugging, its useful to check if Cassandra indeed is picking up the > JVM args supplied to it. Therefore, we should print those on startup. > Patch is attached, and straightforward. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10764) Cassandra Daemon should print JVM arguments
[ https://issues.apache.org/jira/browse/CASSANDRA-10764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Kale updated CASSANDRA-10764: - Description: While debugging, its useful to check if Cassandra indeed is picking up the JVM args supplied to it. Therefore, we should print those on startup. I'll submit a patch shortly. was: Sometimes its useful to check if Cassandra indeed is picking up the JVM args supplied to it. Therefore, we should print those on startup. I'll submit a patch shortly. > Cassandra Daemon should print JVM arguments > --- > > Key: CASSANDRA-10764 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10764 > Project: Cassandra > Issue Type: Improvement > Components: Tools > Environment: PROD >Reporter: Anubhav Kale >Priority: Minor > Fix For: 2.1.12 > > Attachments: 10764.patch > > > While debugging, its useful to check if Cassandra indeed is picking up the > JVM args supplied to it. Therefore, we should print those on startup. > I'll submit a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10764) Cassandra Daemon should print JVM arguments
[ https://issues.apache.org/jira/browse/CASSANDRA-10764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Kale updated CASSANDRA-10764: - Attachment: 10764.patch > Cassandra Daemon should print JVM arguments > --- > > Key: CASSANDRA-10764 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10764 > Project: Cassandra > Issue Type: Improvement > Components: Tools > Environment: PROD >Reporter: Anubhav Kale >Priority: Minor > Fix For: 2.1.12 > > Attachments: 10764.patch > > > Sometimes its useful to check if Cassandra indeed is picking up the JVM args > supplied to it. Therefore, we should print those on startup. > I'll submit a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8152) Cassandra crashes with Native memory allocation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-8152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg reassigned CASSANDRA-8152:
-
Assignee: Ariel Weisberg
> Cassandra crashes with Native memory allocation failure
> ---
>
> Key: CASSANDRA-8152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8152
> Project: Cassandra
> Issue Type: Bug
> Environment: EC2 (i2.xlarge)
>Reporter: Babar Tareen
>Assignee: Ariel Weisberg
>Priority: Minor
> Attachments: db06_hs_err_pid26159.log.zip,
> db_05_hs_err_pid25411.log.zip
>
>
> On a 6 node Cassandra (datastax-community-2.1) cluster running on EC2
> (i2.xlarge) instances, Jvm hosting the cassandra service randomly crashes
> with following error.
> {code}
> #
> # There is insufficient memory for the Java Runtime Environment to continue.
> # Native memory allocation (malloc) failed to allocate 12288 bytes for
> committing reserved memory.
> # Possible reasons:
> # The system is out of physical RAM or swap space
> # In 32 bit mode, the process size limit was hit
> # Possible solutions:
> # Reduce memory load on the system
> # Increase physical memory or swap space
> # Check if swap backing store is full
> # Use 64 bit Java on a 64 bit OS
> # Decrease Java heap size (-Xmx/-Xms)
> # Decrease number of Java threads
> # Decrease Java thread stack sizes (-Xss)
> # Set larger code cache with -XX:ReservedCodeCacheSize=
> # This output file may be truncated or incomplete.
> #
> # Out of Memory Error (os_linux.cpp:2747), pid=26159, tid=140305605682944
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_60-b19) (build
> 1.7.0_60-b19)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.60-b09 mixed mode
> linux-amd64 compressed oops)
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> --- T H R E A D ---
> Current thread (0x08341000): JavaThread "MemtableFlushWriter:2055"
> daemon [_thread_new, id=23336, stack(0x7f9b71c56000,0x7f9b71c97000)]
> Stack: [0x7f9b71c56000,0x7f9b71c97000], sp=0x7f9b71c95820, free
> space=254k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> V [libjvm.so+0x99e7ca] VMError::report_and_die()+0x2ea
> V [libjvm.so+0x496fbb] report_vm_out_of_memory(char const*, int, unsigned
> long, char const*)+0x9b
> V [libjvm.so+0x81d81e] os::Linux::commit_memory_impl(char*, unsigned long,
> bool)+0xfe
> V [libjvm.so+0x81d8dc] os::pd_commit_memory(char*, unsigned long, bool)+0xc
> V [libjvm.so+0x81565a] os::commit_memory(char*, unsigned long, bool)+0x2a
> V [libjvm.so+0x81bdcd] os::pd_create_stack_guard_pages(char*, unsigned
> long)+0x6d
> V [libjvm.so+0x9522de] JavaThread::create_stack_guard_pages()+0x5e
> V [libjvm.so+0x958c24] JavaThread::run()+0x34
> V [libjvm.so+0x81f7f8] java_start(Thread*)+0x108
> {code}
> Changes in cassandra-env.sh settings
> {code}
> MAX_HEAP_SIZE="8G"
> HEAP_NEWSIZE="800M"
> JVM_OPTS="$JVM_OPTS -XX:TargetSurvivorRatio=50"
> JVM_OPTS="$JVM_OPTS -XX:+AggressiveOpts"
> JVM_OPTS="$JVM_OPTS -XX:+UseLargePages"
> {code}
> Writes are about 10K-15K/sec and there are very few reads. Cassandra 2.0.9
> with same settings never crashed. JVM crash logs are attached from two
> machines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8639) Can OOM on CL replay with dense mutations
[ https://issues.apache.org/jira/browse/CASSANDRA-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg reassigned CASSANDRA-8639: - Assignee: Ariel Weisberg > Can OOM on CL replay with dense mutations > - > > Key: CASSANDRA-8639 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8639 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Ariel Weisberg >Priority: Minor > Fix For: 2.1.x > > > If you write dense mutations with many clustering keys, the replay of the CL > can quickly overwhelm a node on startup. This looks to be caused by the fact > we only ensure there are 1000 mutations in flight at a time. but those > mutations could have thousands of cells in them. > A better approach would be to limit the CL replay to the amount of memory in > flight using cell.unsharedHeapSize() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8152) Cassandra crashes with Native memory allocation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-8152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023304#comment-15023304
]
Ariel Weisberg commented on CASSANDRA-8152:
---
You are hitting the max map count. Props to the JVM folks for the info dump.
These mappings are for files that have already been deleted so it could be that
the C* is pack ratting the mapped buffer somewhere.
How reproducible is this? Does it happen to all six nodes? Is it possible to
get a heap dump? Can you get a count of # of files in the various keyspaces,
preferably at the time you produce a heap dump?
> Cassandra crashes with Native memory allocation failure
> ---
>
> Key: CASSANDRA-8152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8152
> Project: Cassandra
> Issue Type: Bug
> Environment: EC2 (i2.xlarge)
>Reporter: Babar Tareen
>Priority: Minor
> Attachments: db06_hs_err_pid26159.log.zip,
> db_05_hs_err_pid25411.log.zip
>
>
> On a 6 node Cassandra (datastax-community-2.1) cluster running on EC2
> (i2.xlarge) instances, Jvm hosting the cassandra service randomly crashes
> with following error.
> {code}
> #
> # There is insufficient memory for the Java Runtime Environment to continue.
> # Native memory allocation (malloc) failed to allocate 12288 bytes for
> committing reserved memory.
> # Possible reasons:
> # The system is out of physical RAM or swap space
> # In 32 bit mode, the process size limit was hit
> # Possible solutions:
> # Reduce memory load on the system
> # Increase physical memory or swap space
> # Check if swap backing store is full
> # Use 64 bit Java on a 64 bit OS
> # Decrease Java heap size (-Xmx/-Xms)
> # Decrease number of Java threads
> # Decrease Java thread stack sizes (-Xss)
> # Set larger code cache with -XX:ReservedCodeCacheSize=
> # This output file may be truncated or incomplete.
> #
> # Out of Memory Error (os_linux.cpp:2747), pid=26159, tid=140305605682944
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_60-b19) (build
> 1.7.0_60-b19)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.60-b09 mixed mode
> linux-amd64 compressed oops)
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> --- T H R E A D ---
> Current thread (0x08341000): JavaThread "MemtableFlushWriter:2055"
> daemon [_thread_new, id=23336, stack(0x7f9b71c56000,0x7f9b71c97000)]
> Stack: [0x7f9b71c56000,0x7f9b71c97000], sp=0x7f9b71c95820, free
> space=254k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> V [libjvm.so+0x99e7ca] VMError::report_and_die()+0x2ea
> V [libjvm.so+0x496fbb] report_vm_out_of_memory(char const*, int, unsigned
> long, char const*)+0x9b
> V [libjvm.so+0x81d81e] os::Linux::commit_memory_impl(char*, unsigned long,
> bool)+0xfe
> V [libjvm.so+0x81d8dc] os::pd_commit_memory(char*, unsigned long, bool)+0xc
> V [libjvm.so+0x81565a] os::commit_memory(char*, unsigned long, bool)+0x2a
> V [libjvm.so+0x81bdcd] os::pd_create_stack_guard_pages(char*, unsigned
> long)+0x6d
> V [libjvm.so+0x9522de] JavaThread::create_stack_guard_pages()+0x5e
> V [libjvm.so+0x958c24] JavaThread::run()+0x34
> V [libjvm.so+0x81f7f8] java_start(Thread*)+0x108
> {code}
> Changes in cassandra-env.sh settings
> {code}
> MAX_HEAP_SIZE="8G"
> HEAP_NEWSIZE="800M"
> JVM_OPTS="$JVM_OPTS -XX:TargetSurvivorRatio=50"
> JVM_OPTS="$JVM_OPTS -XX:+AggressiveOpts"
> JVM_OPTS="$JVM_OPTS -XX:+UseLargePages"
> {code}
> Writes are about 10K-15K/sec and there are very few reads. Cassandra 2.0.9
> with same settings never crashed. JVM crash logs are attached from two
> machines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10754) Failure in QueryPagerTest.multiQueryTest due to assertion in SinglePartitionReadCommand$Group
[
https://issues.apache.org/jira/browse/CASSANDRA-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Knighton reassigned CASSANDRA-10754:
-
Assignee: Joel Knighton
> Failure in QueryPagerTest.multiQueryTest due to assertion in
> SinglePartitionReadCommand$Group
> -
>
> Key: CASSANDRA-10754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10754
> Project: Cassandra
> Issue Type: Bug
>Reporter: Joel Knighton
>Assignee: Joel Knighton
>
> Test failure
> [here|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-9510-3.0-testall/lastCompletedBuild/testReport/org.apache.cassandra.service/QueryPagerTest/multiQueryTest/].
> It's on a 3.0 branch for a issue, but the patch only affects the
> implementation of the {{assassinate}} command and wouldn't affect this unit
> test.
> The assertion failure is in the construction of a Group of
> SinglePartitionReadCommands, where the value of {{nowInSec}} for each command
> does not match.
> I'm not sure if 1) this invariant isn't valuable, 2) this invariant is
> valuable but the unit test does not accurately reflect how a group would be
> created, or 3) this is a genuine bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8794) AntiEntropySessions doesn't show up until after a repair
[ https://issues.apache.org/jira/browse/CASSANDRA-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023281#comment-15023281 ] Nick Bailey commented on CASSANDRA-8794: If that mbean is gone in 3.0, is there still away to get how many repairs are pending? We were specifically using that mbean to try to determine if repair was backing up on a node. > AntiEntropySessions doesn't show up until after a repair > > > Key: CASSANDRA-8794 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8794 > Project: Cassandra > Issue Type: Bug > Components: Observability >Reporter: Peter Halliday >Assignee: Yuki Morishita > > The metric AntiEntropySessions for internal thread pools doesn't actually > show up as an mbean until after a repair is run. This should actually be > displayed before. This also, keeps any cluster that doesn't need repairing > from displaying stats for AntiEntropySessions. The lack of the mbean's > existence until after the repair will cause problem for various monitoring > tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10738) upgrading to v3.0.0 nodes crashes with "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous bytes after map value"
[ https://issues.apache.org/jira/browse/CASSANDRA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023261#comment-15023261 ] cs commented on CASSANDRA-10738: (ignoring the concept of 'unlucky') can you suggest which schema snapshots to restore in this instance? in v2 they are: ../data/system/schema_*/ and in v3 they are ../data/system_schema/indexes-*/ you suggest keyspaces in the 2nd to last comment, but the log indicates indexes. which sstables from v2 map to the equivalent failing system_schema/indexes-*/ tables in v3? or is there a reason you suggest system_schema.keyspaces? > upgrading to v3.0.0 nodes crashes with > "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value" > --- > > Key: CASSANDRA-10738 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10738 > Project: Cassandra > Issue Type: Bug > Environment: ubuntu14, java8 >Reporter: cs >Assignee: Aleksey Yeschenko > Attachments: cassv3.txt > > > upgrading from v.2.2.3 to v3.0.0 > restarting after upgrade crashes with either > ERROR [main] 2015-11-19 20:21:26,511 CassandraDaemon.java:702 - Exception > encountered during startup > org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value > OR > ERROR [main] 2015-11-19 20:01:13,945 CassandraDaemon.java:702 - Exception > encountered during startup > java.lang.IllegalArgumentException: null > see attached debug and schema -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10280) Make DTCS work well with old data
[ https://issues.apache.org/jira/browse/CASSANDRA-10280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023260#comment-15023260 ] Lorina Poland commented on CASSANDRA-10280: --- To change Cassandra documentation for this JIRA, I need to know if the following is correct. Could someone please verify? C*3.1: 1. Change DEFAULT_MAX_SSTABLE_AGE_DAYS to 1000 days. The patch says 365*1000, which is 365,000 days. 2. Deprecate MAX_SSTABLE_AGE_DAYS. Note that it is deprecated. 3. ADD DEFAULT_MAX_WINDOW_SIZE_SECONDS that is what size? The patch looks like 1 day in seconds, or 86,400, but perhaps I misunderstand. > Make DTCS work well with old data > - > > Key: CASSANDRA-10280 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10280 > Project: Cassandra > Issue Type: Sub-task >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.12, 2.2.4, 3.0.1, 3.1 > > > Operational tasks become incredibly expensive if you keep around a long > timespan of data with DTCS - with default settings and 1 year of data, the > oldest window covers about 180 days. Bootstrapping a node with vnodes with > this data layout will force cassandra to compact very many sstables in this > window. > We should probably put a cap on how big the biggest windows can get. We could > probably default this to something sane based on max_sstable_age (ie, say we > can reasonably handle 1000 sstables per node, then we can calculate how big > the windows should be to allow that) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10585) SSTablesPerReadHistogram seems wrong when row cache hit happend
[ https://issues.apache.org/jira/browse/CASSANDRA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-10585: --- Assignee: Ivan Burmistrov > SSTablesPerReadHistogram seems wrong when row cache hit happend > --- > > Key: CASSANDRA-10585 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10585 > Project: Cassandra > Issue Type: Bug >Reporter: Ivan Burmistrov >Assignee: Ivan Burmistrov >Priority: Minor > Fix For: 2.1.x, 2.2.x, 3.0.x > > Attachments: SSTablePerReadHistogram_RowCache-cassandra-2_1.patch, > SSTablePerReadHistogram_RowCache-cassandra-2_2.patch > > > SSTablePerReadHistogram metric now not considers case when row has been read > from row cache. > And so, this metric will have big values even almost all requests processed > by row cache (and without touching SSTables, of course). > So, it seems that correct behavior is to consider that if we read row from > row cache then we read zero SSTables by this request. > The patch at the attachment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7225) cqlsh help for CQL3 is often incorrect and should be modernized
[
https://issues.apache.org/jira/browse/CASSANDRA-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023214#comment-15023214
]
Paulo Motta commented on CASSANDRA-7225:
Tested on windows, linux and text-mode, works fine. Just some minor doc nits to
wrap-up:
* Can you just add a {{browser = C:/Program Files
(x86)/Google/Chrome/Application/chrome.exe %s}} example for setting up google
chrome on WIndows, in addition to mac and linux on {{conf/cqlshrc.sample}} ?
* Extend the --browser help with (or shorter version):
{noformat}
--browser=BROWSER The browser to use to display CQL help, where BROWSER can
be:
- one of the supported browsers in
https://docs.python.org/2/library/webbrowser.html.
- browser path followed by %s, example:
/usr/bin/google-chrome-stable %s
{noformat}
Thanks!
> cqlsh help for CQL3 is often incorrect and should be modernized
> ---
>
> Key: CASSANDRA-7225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7225
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation and Website, Tools
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Trivial
> Labels: cqlsh, doc-impacting
> Fix For: 3.2, 2.2.x
>
> Attachments: 7225-add-cql-docs-to-debian-package.patch,
> 7225-cqlhelp.txt, EXPAND.pdf
>
>
> Just a small line of text in cqlsh "help" command indicates that < is <= and
> > is >= in CQL.
> This is confusing to many people (including me :) ) because I did not expect
> < to return the "equals" portion.
> Please allow distinct behaviours for <, <=, > and >= in CQL queries. Maybe in
> combination with CASSANDRA-5184 and/or CASSANDRA-4914
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10730) periodic timeout errors in dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-10730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023215#comment-15023215
]
Ariel Weisberg commented on CASSANDRA-10730:
If it fails to make a connection maybe dump the output of netstat -tnlp just as
a starting point to see if the server was listening/running.
> periodic timeout errors in dtest
>
>
> Key: CASSANDRA-10730
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10730
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>
> Dtests often fail with connection timeout errors. For example:
> http://cassci.datastax.com/job/cassandra-3.1_dtest/lastCompletedBuild/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/deletion_test/
> {code}
> ('Unable to connect to any servers', {'127.0.0.1':
> OperationTimedOut('errors=Timed out creating connection (10 seconds),
> last_host=None',)})
> {code}
> We've merged a PR to increase timeouts:
> https://github.com/riptano/cassandra-dtest/pull/663
> It doesn't look like this has improved things:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/363/testReport/
> Next steps here are
> * to scrape Jenkins history to see if and how the number of tests failing
> this way has increased (it feels like it has). From there we can bisect over
> the dtests, ccm, or C*, depending on what looks like the source of the
> problem.
> * to better instrument the dtest/ccm/C* startup process to see why the nodes
> start but don't successfully make the CQL port available.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10765) add RangeIterator interface and QueryPlan for SI
Pavel Yaskevich created CASSANDRA-10765: --- Summary: add RangeIterator interface and QueryPlan for SI Key: CASSANDRA-10765 URL: https://issues.apache.org/jira/browse/CASSANDRA-10765 Project: Cassandra Issue Type: Sub-task Components: Local Write-Read Paths Reporter: Pavel Yaskevich Assignee: Pavel Yaskevich Fix For: 3.2 Currently built-in indexes have only one way of handling intersections/unions: pick the highest selectivity predicate and filter on other index expressions. This is not always the most efficient approach. Dynamic query planning based on the different index characteristics would be more optimal. Query Plan should be able to choose how to do intersections, unions based on the metadata provided by indexes (returned by RangeIterator) and RangeIterator would became a base for cross index interactions and should have information such as min/max token, estimate number of wrapped tokens etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-9813) cqlsh column header can be incorrect when no rows are returned
[
https://issues.apache.org/jira/browse/CASSANDRA-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg reassigned CASSANDRA-9813:
Assignee: (was: Adam Holmberg)
> cqlsh column header can be incorrect when no rows are returned
> --
>
> Key: CASSANDRA-9813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aleksey Yeschenko
> Labels: cqlsh
> Fix For: 2.1.x, 2.2.x, 3.x
>
> Attachments: 9813-2.1.txt, Test-for-9813.txt
>
>
> Upon migration, we internally create a pair of surrogate clustering/regular
> columns for compact static tables. These shouldn't be exposed to the user.
> That is, for the table
> {code}
> CREATE TABLE bar (k int, c int, PRIMARY KEY (k)) WITH COMPACT STORAGE;
> {code}
> {{SELECT * FROM bar}} should not be returning this result set:
> {code}
> cqlsh:test> select * from bar;
> c | column1 | k | value
> ---+-+---+---
> (0 rows)
> {code}
> Should only contain the defined {{c}} and {{k}} columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10764) Cassandra Daemon should print JVM arguments
Anubhav Kale created CASSANDRA-10764: Summary: Cassandra Daemon should print JVM arguments Key: CASSANDRA-10764 URL: https://issues.apache.org/jira/browse/CASSANDRA-10764 Project: Cassandra Issue Type: Improvement Components: Tools Environment: PROD Reporter: Anubhav Kale Priority: Minor Fix For: 2.1.12 Sometimes its useful to check if Cassandra indeed is picking up the JVM args supplied to it. Therefore, we should print those on startup. I'll submit a patch shortly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9813) cqlsh column header can be incorrect when no rows are returned
[
https://issues.apache.org/jira/browse/CASSANDRA-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023184#comment-15023184
]
Adam Holmberg commented on CASSANDRA-9813:
--
bq. Also looks like there is no test for this in the driver commit?
It's in a different commit provided by a test engineer:
https://github.com/datastax/python-driver/commit/dcfe67b300cfa60c99bf9f860fc2b716aafce8b4#diff-58ee79e6ce67a66fa7174dd80e457546R164
bq. In the python driver it retrieves the column names by index. Seems a little
fragile to not pull it out via an associative lookup,
There are a number of internal interfaces in this driver that pass compound
data structures. I am aware, but have elected not to make them associative
because their use is limited and well-understood. This should not be a concern
of clients using the driver.
> cqlsh column header can be incorrect when no rows are returned
> --
>
> Key: CASSANDRA-9813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9813
> Project: Cassandra
> Issue Type: Bug
>Reporter: Aleksey Yeschenko
>Assignee: Adam Holmberg
> Labels: cqlsh
> Fix For: 2.1.x, 2.2.x, 3.x
>
> Attachments: 9813-2.1.txt, Test-for-9813.txt
>
>
> Upon migration, we internally create a pair of surrogate clustering/regular
> columns for compact static tables. These shouldn't be exposed to the user.
> That is, for the table
> {code}
> CREATE TABLE bar (k int, c int, PRIMARY KEY (k)) WITH COMPACT STORAGE;
> {code}
> {{SELECT * FROM bar}} should not be returning this result set:
> {code}
> cqlsh:test> select * from bar;
> c | column1 | k | value
> ---+-+---+---
> (0 rows)
> {code}
> Should only contain the defined {{c}} and {{k}} columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9277) SSTableRewriterTest.testFileRemoval fails with test-compression
[ https://issues.apache.org/jira/browse/CASSANDRA-9277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9277: -- Component/s: Testing > SSTableRewriterTest.testFileRemoval fails with test-compression > --- > > Key: CASSANDRA-9277 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9277 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.0 beta 1 > > > openEarly returns null with compression because not enough data has been > written to trigger a flush. Solution is to write more data in the test case > so it flushes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9276) StreamingTransferTest fails under test-compression due to bad assertion
[
https://issues.apache.org/jira/browse/CASSANDRA-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-9276:
--
Component/s: Streaming and Messaging
> StreamingTransferTest fails under test-compression due to bad assertion
> ---
>
> Key: CASSANDRA-9276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9276
> Project: Cassandra
> Issue Type: Test
> Components: Streaming and Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.2.0 beta 1
>
>
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/streaming/compress/CompressedStreamReader.java#L85
> {noformat}
> assert in.getBytesRead() < totalSize;
> {noformat}
> My guess is that total size is the compressed size, not the uncompressed
> size. Remove the assertion and the test passes.
> Total size is calculated with
> {noformat}
> long size = 0;
> // calculate total length of transferring chunks
> for (CompressionMetadata.Chunk chunk : compressionInfo.chunks)
> size += chunk.length + 4; // 4 bytes for CRC
> return size;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8776) nodetool status reports success for missing keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-8776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8776: -- Assignee: Sachin Janani (was: Ariel Weisberg) > nodetool status reports success for missing keyspace > > > Key: CASSANDRA-8776 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8776 > Project: Cassandra > Issue Type: Bug >Reporter: Stuart Bishop >Assignee: Sachin Janani >Priority: Minor > Labels: lhf > Fix For: 2.1.5 > > Attachments: 8776_1.patch > > > 'nodetool status somethinginvalid' will correctly output an error message > that the keyspace does not exist, but still returns a 'success' code of 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9029) Add utility class to support for rate limiting a given log statement
[ https://issues.apache.org/jira/browse/CASSANDRA-9029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9029: -- Component/s: Observability > Add utility class to support for rate limiting a given log statement > > > Key: CASSANDRA-9029 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9029 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.1.6, 2.2.0 beta 1 > > > Add a utility class that can be used in the code to rate limit a given log > statement. This can be used when the log statement is coming from a > performance sensitive place or someplace hit often, and you don't want it to > spam the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9287) CQLSSTableWriterLongTest is failing
[
https://issues.apache.org/jira/browse/CASSANDRA-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-9287:
--
Component/s: Testing
> CQLSSTableWriterLongTest is failing
> ---
>
> Key: CASSANDRA-9287
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9287
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.1.6, 2.2.0 beta 1
>
>
> When I ran it I get an error in setup related the auth keyspace not existing.
> It exists, but it's not in the Schema singleton. Lost half a day on this
> before looking at other usages of StorageService.instance.initServer() and
> finding that the other usages also have
> {noformat}
> SchemaLoader.cleanupAndLeaveDirs();
> Keyspace.setInitialized();
> {noformat}
> first. So some magic was apparently missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9278) LeveledCompactionStrategytest.testGrouperLevels fails with test-compression
[ https://issues.apache.org/jira/browse/CASSANDRA-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9278: -- Component/s: Testing > LeveledCompactionStrategytest.testGrouperLevels fails with test-compression > --- > > Key: CASSANDRA-9278 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9278 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.0 beta 1 > > > Compression causes fewer sstables to be emitted so the test fails when it > goes to look for the tables. Solution is to use entropy for the data so it > doesn't compress. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9376) Investigate using asynchronous logback config for tests to avoid timeouts and reduce test time
[ https://issues.apache.org/jira/browse/CASSANDRA-9376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9376: -- Component/s: Testing > Investigate using asynchronous logback config for tests to avoid timeouts and > reduce test time > -- > > Key: CASSANDRA-9376 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9376 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.0 beta 1 > > > unit tests are run with debug logging which can generate a lot of output. > Every log statement results in a write to the file descriptor so it is not > zippy. > Make the logging async and don't flush for every log statement. Have logback > register a shutdown hook to make sure everything flushes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9403) Experiment with skipping file syncs during unit tests to reduce test time
[ https://issues.apache.org/jira/browse/CASSANDRA-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9403: -- Component/s: Testing > Experiment with skipping file syncs during unit tests to reduce test time > - > > Key: CASSANDRA-9403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9403 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.0 rc1 > > > Some environments have ridiculous outliers for disk syncing. 20 seconds > ridiculous. > Unit tests aren't testing crash safety so it is a pointless exercise. > Instead we could intercept calls to sync files and check whether it looks > like the sync would succeed. Check that the things are not null, mapped, > closed etc. Outside of units tests it can go straight to the regular sync > call. > I would also like to have the disks for unit and dtests mounted with > barrier=0,noatime,nodiratime to further reduce susceptibility to outliers. We > aren't going to recover these nodes if they crash/restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9073) o.a.c.schema.LegacySchemaTablesTest.dropCf may be flapping
[ https://issues.apache.org/jira/browse/CASSANDRA-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9073: -- Component/s: Testing > o.a.c.schema.LegacySchemaTablesTest.dropCf may be flapping > -- > > Key: CASSANDRA-9073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9073 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > Saw this fail on my Linux desktop. It doesn't fail every time. This is > running trunk @ ff5ed7a03f7b9968c0156b05226af67882e5670e > This doesn't fail in cassci -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9271) IndexSummaryManagerTest.testCompactionRace times out periodically
[ https://issues.apache.org/jira/browse/CASSANDRA-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9271: -- Component/s: Testing > IndexSummaryManagerTest.testCompactionRace times out periodically > - > > Key: CASSANDRA-9271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9271 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Trivial > Labels: test-failure > Fix For: 2.1.6, 2.2.0 rc1 > > Attachments: equivalent_tracker_unit_test.txt > > > The issue is that the amount of time the test takes is highly variable to it > being biased towards creating a condition where the test has to retry the > compaction it is attempting. > Solution is to decrease the bias by having > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L2522 > check every millisecond instead of every 100 milliseconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8584) Add rate limited logging of failed trySkipCache calls and commit log lag
[ https://issues.apache.org/jira/browse/CASSANDRA-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8584: -- Component/s: Observability Local Write-Read Paths > Add rate limited logging of failed trySkipCache calls and commit log lag > > > Key: CASSANDRA-8584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8584 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths, Observability >Reporter: Joshua McKenzie >Assignee: Ariel Weisberg >Priority: Trivial > Fix For: 3.0 alpha 1 > > Attachments: 8584_v1.txt, NoSpamLogger.java, nospamlogger.txt > > > Since trySkipCache returns an errno rather than -1 and setting errno like our > other CLibrary calls, it's thread-safe and we could print out more helpful > information if we failed to prompt the kernel to skip the page cache. That > system call should always succeed unless we have an invalid fd as it's free > to ignore us. > Commit log lag is already rate limited by its own implementation. Convert to > use NoSpamLogger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9010) Identify missing test coverage by documented functionality
[ https://issues.apache.org/jira/browse/CASSANDRA-9010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9010: -- Component/s: Testing > Identify missing test coverage by documented functionality > -- > > Key: CASSANDRA-9010 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9010 > Project: Cassandra > Issue Type: Task > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Labels: monthly-release > > The output of this is a spreadsheet that is already being worked on. > [~philipthompson] you can assign to me once you are done and I can do another > pass and review things like unit test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10681) make index building pluggable via IndexBuildTask
[ https://issues.apache.org/jira/browse/CASSANDRA-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023148#comment-15023148 ] Pavel Yaskevich commented on CASSANDRA-10681: - Great, thank you [~beobal]! > make index building pluggable via IndexBuildTask > > > Key: CASSANDRA-10681 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10681 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Labels: sasi > Fix For: 3.2 > > Attachments: 0001-add-table-support-for-multi-table-builds.patch, > 0001-make-index-building-pluggable-via-IndexBuildTask.patch > > > Currently index building assumes one and only way to build all of the indexes > - through SecondaryIndexBuilder - which merges all of the sstables together, > collates columns etc. Such works fine for built-in indexes but not for SASI > since it's attaches to every SSTable individually. We need a "IndexBuildTask" > interface (based on CompactionInfo.Holder) to be returned from Index on > demand to give power to SI interface implementers to decide how build should > work. This might be less effective for CassandraIndex, since this effectively > means that collation will have to be done multiple times on the same data, > but nevertheless is a good compromise for clean interface to outside world. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9023) 2.0.13 write timeouts on driver
[ https://issues.apache.org/jira/browse/CASSANDRA-9023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9023: -- Component/s: Streaming and Messaging > 2.0.13 write timeouts on driver > --- > > Key: CASSANDRA-9023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9023 > Project: Cassandra > Issue Type: Bug > Components: Streaming and Messaging > Environment: For testing using only Single node > hardware configuration as follows: > cpu : > CPU(s):16 > On-line CPU(s) list: 0-15 > Thread(s) per core:2 > Core(s) per socket:8 > Socket(s): 1 > NUMA node(s): 1 > Vendor ID: GenuineIntel > CPU MHz: 2000.174 > L1d cache: 32K > L1i cache: 32K > L2 cache: 256K > L3 cache: 20480K > NUMA node0 CPU(s): 0-15 > OS: > Linux version 2.6.32-504.8.1.el6.x86_64 (mockbu...@c6b9.bsys.dev.centos.org) > (gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC) ) > Disk: There only single disk in Raid i think space is 500 GB used is 5 GB >Reporter: anishek >Assignee: Ariel Weisberg > Attachments: out_system.log > > > Initially asked @ > http://www.mail-archive.com/user@cassandra.apache.org/msg41621.html > Was suggested to post here. > If any more details are required please let me know -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9307) test-long contains tests that are not appropriate for running on every commit
[ https://issues.apache.org/jira/browse/CASSANDRA-9307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9307: -- Component/s: Testing > test-long contains tests that are not appropriate for running on every commit > - > > Key: CASSANDRA-9307 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9307 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 alpha 1 > > > After discussion with Benedict it seems like some of the tests in test-long > are not long unit tests they are burn in tests for components that have not > changed since the tests were introduced. I propose creating a new test-target > called test-burn that isn't part of test-all, commenting test-all to note the > existence of test-burn, and then moving burn in tests to this target. > I'll create a separate JIRA for making sure that test-burn is run regularly > on hardware appropriate for those tests just to prevent bit rot of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9237) Gossip messages subject to head of line blocking by other intra-cluster traffic
[ https://issues.apache.org/jira/browse/CASSANDRA-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9237: -- Component/s: Streaming and Messaging > Gossip messages subject to head of line blocking by other intra-cluster > traffic > --- > > Key: CASSANDRA-9237 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9237 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.1, 3.0 beta 1 > > > Reported as an issue over less than perfect networks like VPNs between data > centers. > Gossip goes over the small message socket where small is 64k which isn't > particularly small. This is done for performance to keep most traffic on one > hot socket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9241) ByteBuffer.array() without ByteBuffer.arrayOffset() + ByteBuffer.position() is a bug
[ https://issues.apache.org/jira/browse/CASSANDRA-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9241: -- Component/s: Testing > ByteBuffer.array() without ByteBuffer.arrayOffset() + ByteBuffer.position() > is a bug > > > Key: CASSANDRA-9241 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9241 > Project: Cassandra > Issue Type: Bug > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Minor > Fix For: 3.2 > > > I found one instance of this on OHCProvider so it make sense to review all > usages since there aren't that many. > Some suspect things: > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/FastByteOperations.java#L197 > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1877 > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/gms/TokenSerializer.java#L40 > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java#L178 > https://github.com/apache/cassandra/blob/trunk/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java#L104 > https://github.com/apache/cassandra/blob/trunk/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java#L543 > https://github.com/apache/cassandra/blob/trunk/tools/stress/src/org/apache/cassandra/stress/operations/predefined/CqlOperation.java#L563 > I made this list off of 8099 so I might have missed some instances on trunk. > FastByteOperations makes me cross eyed so it is worth a second pass to make > sure offsets in byte buffers are handled correctly. > Generally I like to use the full incantation even when I have done things > like allocate the buffer on the stack locally for copy pasta/refactoring > reasons and to make clear to new users how the API is supposed to work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9124) GCInspector logs very different times after CASSANDRA-7638
[
https://issues.apache.org/jira/browse/CASSANDRA-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-9124:
--
Component/s: Observability
> GCInspector logs very different times after CASSANDRA-7638
> --
>
> Key: CASSANDRA-9124
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9124
> Project: Cassandra
> Issue Type: Bug
> Components: Observability
>Reporter: Jeremiah Jordan
>Assignee: Ariel Weisberg
>Priority: Minor
> Fix For: 2.1.6, 2.2.0 beta 1
>
>
> After the GCInspector rewrite in CASSANDRA-7638 the times reported for CMS
> are the full time (including all the concurrent time), not just the stop the
> world pause time. In previous versions we reported just the stop the world
> pause time. This change is kind of scary for someone used to the old logs,
> and is also not as useful. You can't get "how long were things really
> stopped" from the log message any more.
> For example, this is a CMS that got logged in C* 2.1:
> {noformat}
> INFO [Service Thread] 2015-04-03 23:58:37,583 GCInspector.java:142 -
> ConcurrentMarkSweep GC in 12926ms. CMS Old Gen: 5305346280 -> 1106799064;
> Par Eden Space: 223080 -> 158423560; Par Survivor Space: 42081744 -> 51339584
> {noformat}
> And here is the corresponding information for that CMS from the gc log.
> {noformat}
> 2015-04-03T23:58:24.656+: 8064.780: [GC [1 CMS-initial-mark:
> 5181002K(6901760K)] 5222315K(7639040K), 0.0316710 secs] [Times: user=0.03
> sys=0.00, real=0.03 secs]
> 2015-04-03T23:58:24.688+: 8064.812: Total time for which application
> threads were stopped: 0.0324490 seconds
> 2015-04-03T23:58:24.688+: 8064.812: [CMS-concurrent-mark-start]
> 2015-04-03T23:58:26.939+: 8067.062: [CMS-concurrent-mark: 2.176/2.250
> secs] [Times: user=12.94 sys=1.73, real=2.25 secs]
> 2015-04-03T23:58:26.939+: 8067.063: [CMS-concurrent-preclean-start]
> 2015-04-03T23:58:27.209+: 8067.333: [CMS-concurrent-preclean: 0.187/0.270
> secs] [Times: user=1.53 sys=0.15, real=0.28 secs]
> 2015-04-03T23:58:27.210+: 8067.333:
> [CMS-concurrent-abortable-preclean-start]
> 2015-04-03T23:58:27.988+: 8068.112: [CMS-concurrent-abortable-preclean:
> 0.759/0.779 secs] [Times: user=4.07 sys=0.74, real=0.77 secs]
> 2015-04-03T23:58:27.989+: 8068.113: [GC[YG occupancy: 488441 K (737280
> K)]2015-04-03T23:58:27.989+: 8068.113: [Rescan (parallel) , 0.3688960
> secs]2015-04-03T23:58:28.358+: 8068.482: [weak refs processing, 0.0009620
> secs]2015-04-03T23:58:28.359+: 8068.483: [class unloading, 0.0060870
> secs]2015-04-03T23:58:28.365+: 8068.489: [scrub symbol table, 0.0146010
> secs]2015-04-03T23:58:28.380+: 8068.504: [scrub string table, 0.0031270
> secs] [1 CMS-remark: 5231445K(6901760K)] 5719886K(7639040K), 0.3953770 secs]
> [Times: user=2.96 sys=0.00, real=0.39 secs]
> 2015-04-03T23:58:28.385+: 8068.508: Total time for which application
> threads were stopped: 0.3962470 seconds
> 2015-04-03T23:58:28.385+: 8068.509: [CMS-concurrent-sweep-start]
> 2015-04-03T23:58:37.582+: 8077.706: [CMS-concurrent-sweep: 8.661/9.197
> secs] [Times: user=44.80 sys=9.58, real=9.20 secs]
> 2015-04-03T23:58:37.589+: 8077.713: [CMS-concurrent-reset-start]
> 2015-04-03T23:58:37.633+: 8077.757: [CMS-concurrent-reset: 0.044/0.044
> secs] [Times: user=0.19 sys=0.10, real=0.04 secs]
> {noformat}
> The entire CMS took the 12 seconds reported in the GCIspector log message.
> Previously we would have only reported the 0.39 seconds that were spent in
> STW pauses.
> At the least we need to change the log message so that people don't think we
> are still just reporting STW time. But it would be more helpful if we could
> get the STW time and put that into the log message like we had previously.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8789) OutboundTcpConnectionPool should route messages to sockets by size not type
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8789: -- Component/s: Streaming and Messaging > OutboundTcpConnectionPool should route messages to sockets by size not type > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.0 beta 1 > > Attachments: 8789.diff > > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record of what the impact of this change was. > If someone remembers what situations were made better it would be good to > know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8771) Remove commit log segment recycling
[ https://issues.apache.org/jira/browse/CASSANDRA-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8771: -- Component/s: Local Write-Read Paths > Remove commit log segment recycling > --- > > Key: CASSANDRA-8771 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8771 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Labels: commitlog > Fix For: 2.2.0 beta 1 > > > For discussion > Commit log segment recycling introduces a lot of complexity in the existing > code. > CASSANDRA-8729 is a side effect of commit log segment recycling and > addressing it will require memory management code and thread coordination for > memory that the filesystem will no longer handle for us. > There is some discussion about what storage configurations actually benefit > from preallocated files. Fast random access devices like SSDs, or > non-volatile write caches etc. make the distinction not that great. > I haven't measured any difference in throughput for bulk appending vs > overwriting although it was pointed out that I didn't test with concurrent IO > streams. > What would it take to make removing commit log segment recycling acceptable? > Maybe a benchmark on a spinning disk that measures the performance impact of > preallocation when there are other IO streams? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8692) Coalesce intra-cluster network messages
[ https://issues.apache.org/jira/browse/CASSANDRA-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8692: -- Component/s: Streaming and Messaging > Coalesce intra-cluster network messages > --- > > Key: CASSANDRA-8692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8692 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.1.5 > > Attachments: batching-benchmark.png > > > While researching CASSANDRA-8457 we found that it is effective and can be > done without introducing additional latency at low concurrency/throughput. > The patch from that was used and found to be useful in a real life scenario > so I propose we implement this in 2.1 in addition to 3.0. > The change set is a single file and is small enough to be reviewable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10763) jmxmetrics_test.TestJMXMetrics.begin_test is failing
Philip Thompson created CASSANDRA-10763:
---
Summary: jmxmetrics_test.TestJMXMetrics.begin_test is failing
Key: CASSANDRA-10763
URL: https://issues.apache.org/jira/browse/CASSANDRA-10763
Project: Cassandra
Issue Type: Sub-task
Components: Testing
Reporter: Philip Thompson
Fix For: 3.0.1, 3.1
jmxmetrics_test.TestJMXMetrics.begin_test is failing against 3.0
The error message is
{code}
org.apache.cassandra.metrics:type=Table,name=CompressionRatio has a before
value of 0.821687586749 and after value of 0.822466108603 and did not decrement
{code}
The test expects that mbean to decrease over the course of the test, but it
does not. I do not know if this is a bug, a change in behavior, or simply an
incorrect assumption by the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8677) rpc_interface and listen_interface generate NPE on startup when specified interface doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-8677:
--
Component/s: Configuration
> rpc_interface and listen_interface generate NPE on startup when specified
> interface doesn't exist
> -
>
> Key: CASSANDRA-8677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8677
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.1.3, 2.2.0 beta 1
>
> Attachments: 8677-2.1.patch, 8677.patch
>
>
> This is just a buggy UI bit.
> Initially the error I got was this which is redundant and not well formatted.
> {noformat}
> ERROR 20:12:55 Exception encountered during startup
> java.lang.ExceptionInInitializerError: null
> Fatal configuration error; unable to start. See log for stacktrace.
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:108)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
> [main/:na]
> java.lang.ExceptionInInitializerError: null
> Fatal configuration error; unable to start. See log for stacktrace.
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:108)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
> Exception encountered during startup: null
> Fatal configuration error; unable to start. See log for stacktrace.
> ERROR 20:12:55 Exception encountered during startup
> java.lang.ExceptionInInitializerError: null
> Fatal configuration error; unable to start. See log for stacktrace.
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:108)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
> [main/:na]
> {noformat}
> This has no description of the error that occurred. After logging the
> exception.
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:347)
> ~[main/:na]
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:102)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
> [main/:na]
> {noformat}
> Exceptions thrown in the DatabaseDescriptor should log in a useful way.
> This particular error should generate a message without a stack trace since
> it is easily recognized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10761) Possible regression of CASSANDRA-9201
[
https://issues.apache.org/jira/browse/CASSANDRA-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-10761:
Fix Version/s: 2.2.x
> Possible regression of CASSANDRA-9201
> -
>
> Key: CASSANDRA-10761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10761
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Philip Thompson
> Fix For: 3.0.1, 3.1, 2.2.x
>
> Attachments: 10761-logs.tar.gz
>
>
> Some dtests like
> {{consistency_test.TestAccuracy.test_network_topology_strategy_each_quorum_counters}}
> are failing with the follow auth related assertion exception
> {code}
> [node6 ERROR] java.lang.AssertionError:
> org.apache.cassandra.exceptions.InvalidRequestException: unconfigured table
> roles
> at
> org.apache.cassandra.auth.CassandraRoleManager.prepare(CassandraRoleManager.java:450)
> at
> org.apache.cassandra.auth.CassandraRoleManager.setup(CassandraRoleManager.java:144)
> at
> org.apache.cassandra.service.StorageService.doAuthSetup(StorageService.java:1036)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:984)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:708)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:579)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:345)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:561)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689)
> Caused by: org.apache.cassandra.exceptions.InvalidRequestException:
> unconfigured table roles
> at
> org.apache.cassandra.thrift.ThriftValidation.validateColumnFamily(ThriftValidation.java:114)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:757)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:752)
> at
> org.apache.cassandra.auth.CassandraRoleManager.prepare(CassandraRoleManager.java:446)
> ... 8 more
> {code}
> This looks very similar to CASSANDRA-9201.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage
[
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-8670:
--
Component/s: Streaming and Messaging
Local Write-Read Paths
Compaction
> Large columns + NIO memory pooling causes excessive direct memory usage
> ---
>
> Key: CASSANDRA-8670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction, Local Write-Read Paths, Streaming and
> Messaging
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 2.2.0 beta 1
>
> Attachments: OutputStreamBench.java, largecolumn_test.py
>
>
> If you provide a large byte array to NIO and ask it to populate the byte
> array from a socket it will allocate a thread local byte buffer that is the
> size of the requested read no matter how large it is. Old IO wraps new IO for
> sockets (but not files) so old IO is effected as well.
> Even If you are using Buffered{Input | Output}Stream you can end up passing a
> large byte array to NIO. The byte array read method will pass the array to
> NIO directly if it is larger than the internal buffer.
> Passing large cells between nodes as part of intra-cluster messaging can
> cause the NIO pooled buffers to quickly reach a high watermark and stay
> there. This ends up costing 2x the largest cell size because there is a
> buffer for input and output since they are different threads. This is further
> multiplied by the number of nodes in the cluster - 1 since each has a
> dedicated thread pair with separate thread locals.
> Anecdotally it appears that the cost is doubled beyond that although it isn't
> clear why. Possibly the control connections or possibly there is some way in
> which multiple
> Need a workload in CI that tests the advertised limits of cells on a cluster.
> It would be reasonable to ratchet down the max direct memory for the test to
> trigger failures if a memory pooling issue is introduced. I don't think we
> need to test concurrently pulling in a lot of them, but it should at least
> work serially.
> The obvious fix to address this issue would be to read in smaller chunks when
> dealing with large values. I think small should still be relatively large (4
> megabytes) so that code that is reading from a disk can amortize the cost of
> a seek. It can be hard to tell what the underlying thing being read from is
> going to be in some of the contexts where we might choose to implement
> switching to reading chunks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8614: -- Component/s: Local Write-Read Paths Compaction > Select optimal CRC32 implementation at runtime > -- > > Key: CASSANDRA-8614 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8614 > Project: Cassandra > Issue Type: Improvement > Components: Compaction, Local Write-Read Paths >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Labels: performance > Attachments: 8614.patch, CRC32.class, Sample.java > > > JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec > per core in my quick and dirty test. PureJavaCRC32 is < 800 megabytes/sec if > I recall and it has a lookup table that evicts random cache lines every time > it runs. > In order to capture the benefit of that when it is available we can select a > CRC32 implementation at startup in a static block. > If JDK 8 is not what is running we can fall back to the existing > PureJavaCRC32 implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7404) Use direct i/o for sequential operations (compaction/streaming)
[ https://issues.apache.org/jira/browse/CASSANDRA-7404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-7404: -- Component/s: Local Write-Read Paths Compaction > Use direct i/o for sequential operations (compaction/streaming) > --- > > Key: CASSANDRA-7404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7404 > Project: Cassandra > Issue Type: Improvement > Components: Compaction, Local Write-Read Paths >Reporter: Jason Brown >Assignee: Ariel Weisberg > Labels: performance > > Investigate using linux's direct i/o for operations where we read > sequentially through a file (repair and bootstrap streaming, compaction > reads, and so on). Direct i/o does not go through the kernel page page, so it > should leave the hot cache pages used for live reads unaffected. > Note: by using direct i/o, we will probably take a performance hit on reading > the file we're sequentially scanning through (that is, compactions may get > slower), but the goal of this ticket is to limit the impact of these > background tasks on the main read/write functionality. Of course, I'll > measure any perf hit that is incurred, and see if there's any mechanisms to > mitigate it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6976) Determining replicas to query is very slow with large numbers of nodes or vnodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-6976: -- Component/s: Local Write-Read Paths > Determining replicas to query is very slow with large numbers of nodes or > vnodes > > > Key: CASSANDRA-6976 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6976 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Benedict >Assignee: Ariel Weisberg > Labels: performance > Attachments: GetRestrictedRanges.java, jmh_output.txt, > jmh_output_murmur3.txt, make_jmh_work.patch > > > As described in CASSANDRA-6906, this can be ~100ms for a relatively small > cluster with vnodes, which is longer than it will spend in transit on the > network. This should be much faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6060) Remove internal use of Strings for ks/cf names
[ https://issues.apache.org/jira/browse/CASSANDRA-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-6060: -- Component/s: Streaming and Messaging > Remove internal use of Strings for ks/cf names > -- > > Key: CASSANDRA-6060 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6060 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging >Reporter: Jonathan Ellis >Assignee: Ariel Weisberg > Labels: performance > > We toss a lot of Strings around internally, including across the network. > Once a request has been Prepared, we ought to be able to encode these as int > ids. > Unfortuntely, we moved from int to uuid in CASSANDRA-3794, which was a > reasonable move at the time, but a uuid is a lot bigger than an int. Now > that we have CAS we can allow concurrent schema updates while still using > sequential int IDs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4139) Add varint encoding to Messaging service
[ https://issues.apache.org/jira/browse/CASSANDRA-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-4139: -- Component/s: Streaming and Messaging > Add varint encoding to Messaging service > > > Key: CASSANDRA-4139 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4139 > Project: Cassandra > Issue Type: Sub-task > Components: Streaming and Messaging >Reporter: Vijay >Assignee: Ariel Weisberg > Attachments: 0001-CASSANDRA-4139-v1.patch, > 0001-CASSANDRA-4139-v2.patch, 0001-CASSANDRA-4139-v4.patch, > 0002-add-bytes-written-metric.patch, 4139-Test.rtf, > ASF.LICENSE.NOT.GRANTED--0001-CASSANDRA-4139-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10424) Altering base table column with materialized view causes unexpected server error.
[
https://issues.apache.org/jira/browse/CASSANDRA-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10424:
---
Component/s: Coordination
> Altering base table column with materialized view causes unexpected server
> error.
> -
>
> Key: CASSANDRA-10424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10424
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
> Environment: cassandra-3.0.0-rc1, with python driver 3.0-alpha
>Reporter: Greg Bestland
>Assignee: Carl Yeksigian
> Fix For: 3.0.0 rc2
>
>
> When attempting to alter column type of base table which has a corresponding
> materialized view we get an exception from the server.
> Steps to reproduce.
> 1. Create a base table
> {code}
> CREATE TABLE test.scores(
> user TEXT,
> game TEXT,
> year INT,
> month INT,
> day INT,
> score TEXT,
> PRIMARY KEY (user, game, year, month, day)
> )
> {code}
> 2. Create a corresponding materialized view
> {code}
> CREATE MATERIALIZED VIEW test.monthlyhigh AS
> SELECT game, year, month, score, user, day FROM test.scores
> WHERE game IS NOT NULL AND year IS NOT NULL AND month IS NOT
> NULL AND score IS NOT NULL AND user IS NOT NULL AND day IS NOT NULL
> PRIMARY KEY ((game, year, month), score, user, day)
> WITH CLUSTERING ORDER BY (score DESC, user ASC, day ASC)
> {code}
> 3. Attempt to Alter the base table
> {code}
> ALTER TABLE test.scores ALTER score TYPE blob
> {code}
> In the python driver we see the following exception returned from the server
> {code}
> Ignoring schedule_unique for already-scheduled task: ( ControlConnection.refresh_schema of object at 0x100f72c50>>, (), (('keyspace', 'test'), ('target_type',
> 'KEYSPACE'), ('change_type', 'UPDATED')))
> Traceback (most recent call last):
> File "", line 1, in
> File "./cassandra/cluster.py", line 1623, in execute
> result = future.result()
> File "./cassandra/cluster.py", line 3205, in result
> raise self._final_exception
> cassandra.protocol.ServerError: message="java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.cassandra.exceptions.ConfigurationException: Column family
> comparators do not match or are not compatible (found ClusteringComparator;
> expected ClusteringComparator).">
> {code}
> On the server I see the following stack trace
> {code}
> INFO [MigrationStage:1] 2015-09-30 11:45:47,457 ColumnFamilyStore.java:825 -
> Enqueuing flush of keyspaces: 512 (0%) on-heap, 0 (0%) off-heap
> INFO [MemtableFlushWriter:11] 2015-09-30 11:45:47,457 Memtable.java:362 -
> Writing Memtable-keyspaces@1714565887(0.146KiB serialized bytes, 1 ops, 0%/0%
> of on/off-heap limit)
> INFO [MemtableFlushWriter:11] 2015-09-30 11:45:47,463 Memtable.java:395 -
> Completed flushing
> /Users/gregbestland/.ccm/tests/node1/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/ma-54-big-Data.db
> (0.109KiB) for commitlog position ReplayPosition(segmentId=1443623481894,
> position=9812)
> INFO [MigrationStage:1] 2015-09-30 11:45:47,472 ColumnFamilyStore.java:825 -
> Enqueuing flush of columns: 877 (0%) on-heap, 0 (0%) off-heap
> INFO [MemtableFlushWriter:12] 2015-09-30 11:45:47,472 Memtable.java:362 -
> Writing Memtable-columns@771367282(0.182KiB serialized bytes, 1 ops, 0%/0% of
> on/off-heap limit)
> INFO [MemtableFlushWriter:12] 2015-09-30 11:45:47,478 Memtable.java:395 -
> Completed flushing
> /Users/gregbestland/.ccm/tests/node1/data/system_schema/columns-24101c25a2ae3af787c1b40ee1aca33f/ma-51-big-Data.db
> (0.107KiB) for commitlog position ReplayPosition(segmentId=1443623481894,
> position=9812)
> INFO [MigrationStage:1] 2015-09-30 11:45:47,490 ColumnFamilyStore.java:825 -
> Enqueuing flush of views: 2641 (0%) on-heap, 0 (0%) off-heap
> INFO [MemtableFlushWriter:11] 2015-09-30 11:45:47,490 Memtable.java:362 -
> Writing Memtable-views@1740452585(0.834KiB serialized bytes, 1 ops, 0%/0% of
> on/off-heap limit)
> INFO [MemtableFlushWriter:11] 2015-09-30 11:45:47,496 Memtable.java:395 -
> Completed flushing
> /Users/gregbestland/.ccm/tests/node1/data/system_schema/views-9786ac1cdd583201a7cdad556410c985/ma-22-big-Data.db
> (0.542KiB) for commitlog position ReplayPosition(segmentId=1443623481894,
> position=9812)
> ERROR [MigrationStage:1] 2015-09-30 11:45:47,507 CassandraDaemon.java:195 -
> Exception in thread Thread[MigrationStage:1,5,main]
> org.apache.cassandra.exceptions.ConfigurationException: Column family
> comparators do not match
[jira] [Updated] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
[ https://issues.apache.org/jira/browse/CASSANDRA-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8684: -- Component/s: Local Write-Read Paths > Replace usage of Adler32 with CRC32 > --- > > Key: CASSANDRA-8684 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8684 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 beta 1 > > Attachments: CRCBenchmark.java, PureJavaCrc32.java, Sample.java > > > I could not find a situation in which Adler32 outperformed PureJavaCrc32 much > less the intrinsic from Java 8. For small allocations PureJavaCrc32 was much > faster probably due to the JNI overhead of invoking the native Adler32 > implementation where the array has to be allocated and copied. > I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as > well as a c3.8xlarge running Ubuntu 14.04. > I think it makes sense to stop using Adler32 when generating new checksums. > c3.8xlarge, results are time in milliseconds, lower is better > ||Allocation size|Adler32|CRC32|PureJavaCrc32|| > |64|47636|46075|25782| > |128|36755|36712|23782| > |256|31194|32211|22731| > |1024|27194|28792|22010| > |1048576|25941|27807|21808| > |536870912|25957|27840|21836| > i5 > ||Allocation size|Adler32|CRC32|PureJavaCrc32|| > |64|50539|50466|26826| > |128|37092|38533|24553| > |256|30630|32938|23459| > |1024|26064|29079|22592| > |1048576|24357|27911|22481| > |536870912|24838|28360|22853| > Another fun fact. Performance of the CRC32 intrinsic appears to double from > Sandy Bridge -> Haswell. Unless I am measuring something different when going > from Linux/Sandy to Haswell/OS X. > The intrinsic/JDK 8 implementation also operates against DirectByteBuffers > better and coding against the wrapper will get that boost when run with Java > 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10147) Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
[
https://issues.apache.org/jira/browse/CASSANDRA-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10147:
---
Component/s: CQL
> Base table PRIMARY KEY can be assumed to be NOT NULL in MV creation
> ---
>
> Key: CASSANDRA-10147
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10147
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0 beta 2
>
>
> {code}
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> username text,
> email text,
> age int
> );
> CREATE MATERIALIZED VIEW users_by_username AS SELECT * FROM users WHERE
> username IS NOT NULL PRIMARY KEY (username, id);
> InvalidRequest: code=2200 [Invalid query] message="Primary key column 'id' is
> required to be filtered by 'IS NOT NULL'"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9432) SliceQueryFilterWithTombstonesTest.testExpiredTombstones is failing infrequently
[ https://issues.apache.org/jira/browse/CASSANDRA-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9432: -- Component/s: Testing > SliceQueryFilterWithTombstonesTest.testExpiredTombstones is failing > infrequently > > > Key: CASSANDRA-9432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9432 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.0 rc1 > > > I have only seen one instance of this, but haven't looked hard. > http://cassci.datastax.com/view/trunk/job/trunk_testall/98/testReport/junit/org.apache.cassandra.cql3/SliceQueryFilterWithTombstonesTest/testExpiredTombstones/ > I have it running in a loop and it's not reproducing. For now I just want to > log the exception that is causing the failure. Need the ticket to to tie the > commit to. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9560) Changing durable_writes on a keyspace is only applied after restart of node
[
https://issues.apache.org/jira/browse/CASSANDRA-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-9560:
--
Component/s: Distributed Metadata
> Changing durable_writes on a keyspace is only applied after restart of node
> ---
>
> Key: CASSANDRA-9560
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9560
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
> Environment: Single node
>Reporter: Fred A
>Assignee: Carl Yeksigian
> Fix For: 2.0.17, 2.1.8, 2.2.0 rc2
>
>
> When mutations for a column family is about to be applied, the cached
> instance of the keyspace metadata is read. But the schema mutation for
> durable_writes hasn't been applied to this cached instance.
> I'm not too familiar with the codebase but after some debugging (2.1.3), it's
> somehow related to:
> {code:title=org.apache.cassandra.db.Mutation.java|borderStyle=solid}
> public void apply()
> {
>Keyspace ks = Keyspace.open(keyspaceName);
> ks.apply(this, ks.metadata.durableWrites);
> }
> {code}
> Where a cached instance of the keyspace is opened but it's metadata hasn't
> been updated with the earlier applied durable_writes mutation, since it seems
> that the cached keyspace instance is lazily build at startup but after that,
> never updated. I'm also a little bit concerned if other values in the cached
> keyspace instance suffers from the same issue, e.g. replication_factor...
> I've seen the same issue in 2.1.5 and the only way to resolve this issue is
> to restart the node to let the keyspace instance cache reload from disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9383) isPure is no longer used in Functions
[
https://issues.apache.org/jira/browse/CASSANDRA-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-9383:
--
Component/s: CQL
> isPure is no longer used in Functions
> -
>
> Key: CASSANDRA-9383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9383
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
> Fix For: 2.2.0 beta 1
>
> Attachments: 9383-trunk-v2.txt, 9383-trunk.txt
>
>
> After CASSANDRA-9037, we don't use {{Function.isPure}} anymore. Since we have
> {{isDeterministic}} in the UDF schema, we should remove this before we
> release 2.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9286) Add Keyspace/Table details to CollectionType.java error message
[
https://issues.apache.org/jira/browse/CASSANDRA-9286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-9286:
--
Component/s: Observability
> Add Keyspace/Table details to CollectionType.java error message
> ---
>
> Key: CASSANDRA-9286
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9286
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability
>Reporter: sequoyha pelletier
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 2.0.15, 2.1.6
>
> Attachments: 9286-2.0-v2.txt, 9286-2.0.txt, 9286-2.1-v2.txt,
> 9286-2.1.txt
>
>
> The error message for too many element in a collection does not give keyspace
> or column family information. This makes it a pain point to try to determine
> which table is the offending table.
> Example Error message:
> {noformat}
> ERROR [Native-Transport-Requests:809453] 2015-04-23 22:48:21,189
> CollectionType.java (line 116) Detected collection with 136234 elements, more
> than the 65535 limit. Only the first 65535 elements will be returned to the
> client. Please see http://cassandra.apache.org/doc/cql3/CQL.html#collections
> for more details.
> {noformat}
> Currently, to try to pinpoint the table in question. We need to trace all
> requests and then try to match up the timestamps in the CQL tracing session
> with the log timestamps to try and match. If prepared statements are used,
> this is a dead end due to the logged tracing information missing the query.
> In which case, we have to look at other 3rd party methods for capturing the
> queries to try and match up. This is extremely tedious when many tables have
> collections and a high number of ops against them.
> Requesting that the error contain the keyspace.table name in the error
> message.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9462) ViewTest.sstableInBounds is failing
[ https://issues.apache.org/jira/browse/CASSANDRA-9462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9462: -- Component/s: Streaming and Messaging Local Write-Read Paths > ViewTest.sstableInBounds is failing > --- > > Key: CASSANDRA-9462 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9462 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths, Streaming and Messaging >Reporter: Benedict >Assignee: Ariel Weisberg > Fix For: 2.2.1, 3.0 beta 1 > > > CASSANDRA-8568 introduced new tests to cover what was DataTracker > functionality in 2.1, and is now covered by the lifecycle package. This > particular test indicates this method does not fulfil the expected contract, > namely that more sstables are returned than should be. > However while looking into it I noticed it also likely has a bug (which I > have not updated the test to cover) wherein a wrapped range will only yield > the portion at the end of the token range, not the beginning. It looks like > we may have call sites using this function that do not realise this, so it > could be a serious bug, especially for repair. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9310) Table change response returns as keyspace change response
[
https://issues.apache.org/jira/browse/CASSANDRA-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-9310:
--
Component/s: CQL
> Table change response returns as keyspace change response
> -
>
> Key: CASSANDRA-9310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9310
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: C* 1.2.19 and 2.0.14 | python-driver master (upcoming v.
> 2.6)
>Reporter: Kishan Karunaratne
>Assignee: Carl Yeksigian
> Fix For: 2.0.16, 2.1.6
>
> Attachments: 9310-1.2.txt, 9310-2.0.txt, 9310-2.1-v2.txt, 9310-2.1.txt
>
>
> When an index is dropped, its existence is still persisted across the
> keyspace metadata. This happens because the response to drop the index from
> the metadata is never received, as a keyspace change response is
> (incorrectly) received by the driver instead of a table change response.
> Related to PYTHON-241: https://datastax-oss.atlassian.net/browse/PYTHON-241
> Test:
> {noformat}
> self.session.execute("CREATE TABLE %s (k int PRIMARY KEY, a int)" %
> self.table_name)
> ks_meta = self.cluster.metadata.keyspaces[self.keyspace_name]
> table_meta = ks_meta.tables[self.table_name]
> self.assertNotIn('a_idx', ks_meta.indexes)
> self.assertNotIn('b_idx', ks_meta.indexes)
> self.assertNotIn('a_idx', table_meta.indexes)
> self.assertNotIn('b_idx', table_meta.indexes)
> self.session.execute("CREATE INDEX a_idx ON %s (a)" % self.table_name)
> self.session.execute("ALTER TABLE %s ADD b int" % self.table_name)
> self.session.execute("CREATE INDEX b_idx ON %s (b)" % self.table_name)
> ks_meta = self.cluster.metadata.keyspaces[self.keyspace_name]
> table_meta = ks_meta.tables[self.table_name]
> self.assertIsInstance(ks_meta.indexes['a_idx'], IndexMetadata)
> self.assertIsInstance(ks_meta.indexes['b_idx'], IndexMetadata)
> self.assertIsInstance(table_meta.indexes['a_idx'], IndexMetadata)
> self.assertIsInstance(table_meta.indexes['b_idx'], IndexMetadata)
> # both indexes updated when index dropped
> self.session.execute("DROP INDEX a_idx")
> ks_meta = self.cluster.metadata.keyspaces[self.keyspace_name]
> table_meta = ks_meta.tables[self.table_name]
> self.assertNotIn('a_idx', ks_meta.indexes)
> {noformat}
> Output:
> {noformat}
> AssertionError: 'a_idx' unexpectedly found in {u'b_idx':
> , u'a_idx':
> }
> {noformat}
> Debug log:
> {noformat}
> cassandra.connection: DEBUG: Message pushed from server:
> event_args={'keyspace': u'index_map_tests', 'change_type': u'CREATED',
> 'table': u''}, trace_id=None)>
> cassandra.cluster: DEBUG: Refreshing schema in response to schema change.
> Keyspace: index_map_tests; Table: , Type: None
> cassandra.cluster: DEBUG: [control connection] Waiting for schema agreement
> cassandra.cluster: DEBUG: [control connection] Schemas mismatched, trying
> again
> cassandra.cluster: DEBUG: [control connection] Schemas mismatched, trying
> again
> cassandra.cluster: DEBUG: [control connection] Schemas mismatched, trying
> again
> cassandra.cluster: DEBUG: [control connection] Schemas match
> cassandra.cluster: DEBUG: [control connection] Waiting for schema agreement
> cassandra.cluster: DEBUG: [control connection] Fetched keyspace info for
> index_map_tests, rebuilding metadata
> cassandra.cluster: DEBUG: [control connection] Schemas match
> cassandra.cluster: DEBUG: [control connection] Fetched keyspace info for
> index_map_tests, rebuilding metadata
> cassandra.connection: DEBUG: Message pushed from server:
> event_args={'keyspace': u'index_map_tests', 'change_type': u'CREATED',
> 'table': u'test_index_updates'}, trace_id=None)>
> cassandra.cluster: DEBUG: Refreshing schema in response to schema change.
> Keyspace: index_map_tests; Table: test_index_updates, Type: None
> cassandra.cluster: DEBUG: [control connection] Waiting for schema agreement
> cassandra.cluster: DEBUG: [control connection] Schemas mismatched, trying
> again
> cassandra.cluster: DEBUG: [control connection] Schemas mismatched, trying
> again
> cassandra.cluster: DEBUG: [control connection] Schemas match
> cassandra.cluster: DEBUG: [control connection] Fetched table info for
> index_map_tests.test_index_updates, rebuilding metadata
> cassandra.connection: DEBUG: Message pushed from server:
> event_args={'keyspace': u'index_map_tests', 'change_type': u'UPDATED',
> 'table': u'test_index_updates'}, trace_id=None)>
> cassandra.cluster: DEBUG: Ignoring schedule_unique for already-scheduled
> task: ( >,
> (u'index_map_tests', u'test_index_updates', None))
> cassandra.cluster: DEBUG: Refreshing schema in response to schema change.
> Keyspace: index_map_tests; Table: test_index_updates, Type: None
> cassandra.cluster: DEBUG: [control connection] Waiting for schema agreeme
[jira] [Updated] (CASSANDRA-9057) index validation fails for non-indexed column
[
https://issues.apache.org/jira/browse/CASSANDRA-9057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-9057:
--
Component/s: CQL
> index validation fails for non-indexed column
> -
>
> Key: CASSANDRA-9057
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9057
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Eric Evans
>Assignee: Carl Yeksigian
> Fix For: 2.1.6
>
> Attachments: 9057-2.1-v2.txt, 9057-2.1-v3.txt, 9057-2.1.txt,
> 9057-trunk.txt
>
>
> On 2.1.3, updates are failing with an InvalidRequestException when an
> unindexed column is greater than the maximum allowed for indexed entries.
> {noformat}
> ResponseError: Can't index column value of size 1483409 for index null on
> local_group_default_T_parsoid_html.data
> {noformat}
> In this case, the update _does_ include a 1483409 byte column value, but it
> is for a column that is not indexed, (the single indexed column is < 32
> bytes), presumably this is why
> {{cfm.getColumnDefinition(cell.name()).getIndexName()}} returns {{null}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9463) ant test-all results incomplete when parsed
[ https://issues.apache.org/jira/browse/CASSANDRA-9463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9463: -- Component/s: Testing > ant test-all results incomplete when parsed > --- > > Key: CASSANDRA-9463 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9463 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Michael Shuler >Assignee: Ariel Weisberg > Fix For: 2.2.0 rc1, 3.0 alpha 1 > > > trunk `ant test` - 1,196 total tests > trunk `ant test-all` - 1,353 total tests > `ant test-all` runs > "test,long-test,test-compression,pig-test,test-clientutil-jar", so we should > be getting 1196*2 (test, test-compresssion) + N (long-test) + 24 (pig-test) + > N (test-clientutil-jar) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8883) Percentile computation should use ceil not floor in EstimatedHistogram
[
https://issues.apache.org/jira/browse/CASSANDRA-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-8883:
--
Component/s: Observability
> Percentile computation should use ceil not floor in EstimatedHistogram
> --
>
> Key: CASSANDRA-8883
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8883
> Project: Cassandra
> Issue Type: Bug
> Components: Observability
>Reporter: Chris Lohfink
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 2.1.5
>
> Attachments: 8883-2.1.txt
>
>
> When computing the pcount Cassandra uses floor and the comparison with
> elements is >= so given a simple example of there being a total of five
> elements
> {code}
> // data
> [1, 1, 1, 1, 1]
> // offsets
> [1, 2, 3, 4, 5]
> {code}
> Cassandra would report the 50th percentile as 2. While 3 is the more
> expected value. As a comparison using numpy
> {code}
> import numpy as np
> np.percentile(np.array([1, 2, 3, 4, 5]), 50)
> ==> 3.0
> {code}
> The percentiles was added in CASSANDRA-4022 but is now used a lot in metrics
> Cassandra reports. I think it should error on the side on overestimating
> instead of underestimating.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8930) Add a warn notification for clients
[ https://issues.apache.org/jira/browse/CASSANDRA-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-8930: -- Component/s: CQL > Add a warn notification for clients > --- > > Key: CASSANDRA-8930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8930 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Carl Yeksigian >Assignee: Carl Yeksigian > Labels: client-impacting, protocolv4 > Fix For: 2.2.0 beta 1 > > Attachments: 8930-trunk-v2.txt, 8930-trunk.txt, 8930-v3.txt > > > Currently, if a query generates a warning, it is going to be logged server > side. If the person writing the query is not the admin, that warning isn't > going to have an impact on the query, and we're just going to fill up the > server logs. > We should push these warnings back to the client so the driver users can make > necessary changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9485) RangeTombstoneListTest.addAllRandomTest failed on trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9485: -- Component/s: Local Write-Read Paths > RangeTombstoneListTest.addAllRandomTest failed on trunk > --- > > Key: CASSANDRA-9485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9485 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Minor > Fix For: 2.0.16, 2.1.6, 2.2.0 rc1 > > Attachments: RangeTombstoneListTest.java > > > http://cassci.datastax.com/job/trunk_utest/201/testReport/org.apache.cassandra.db/RangeTombstoneListTest/addAllRandomTest/ > The test is also broken for reproducibility. It doesn't print the seed used > for the RNG so it isn't possible to take a failing run and reproduce it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8011) Fail on large batch sizes
[
https://issues.apache.org/jira/browse/CASSANDRA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-8011:
--
Component/s: Configuration
> Fail on large batch sizes
> --
>
> Key: CASSANDRA-8011
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8011
> Project: Cassandra
> Issue Type: Improvement
> Components: Configuration
>Reporter: Patrick McFadin
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 2.2.0 beta 1
>
> Attachments: 8011-trunk-v2.txt, 8011-trunk-v3.txt, 8011-trunk.txt
>
>
> Related to https://issues.apache.org/jira/browse/CASSANDRA-6487
> Just education alone is not going to stop some of the largest batch sizes
> from being used. Just like we have a tombstone fail threshold, I propose that
> we have a batch size fail threshold.
> Maybe 10x warn?
> {{batch_fail_threshold: 50k}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7991) RowIndexEntryTest and SelectWithTokenFunctionTest fail in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-7991: -- Component/s: Testing > RowIndexEntryTest and SelectWithTokenFunctionTest fail in trunk > --- > > Key: CASSANDRA-7991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7991 > Project: Cassandra > Issue Type: Bug > Components: Testing >Reporter: Carl Yeksigian >Assignee: Carl Yeksigian > Fix For: 2.2.0 beta 1 > > Attachments: 7991-trunk.txt > > > org.apache.cassandra.db.RowIndexEntryTest and > org.apache.cassandra.cql3.SelectWithTokenFunctionTest fail consistently on > trunk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7409) Allow multiple overlapping sstables in L1
[ https://issues.apache.org/jira/browse/CASSANDRA-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-7409: -- Component/s: Compaction > Allow multiple overlapping sstables in L1 > - > > Key: CASSANDRA-7409 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7409 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Carl Yeksigian >Assignee: Carl Yeksigian > Labels: compaction > Fix For: 3.x > > > Currently, when a normal L0 compaction takes place (not STCS), we take up to > MAX_COMPACTING_L0 L0 sstables and all of the overlapping L1 sstables and > compact them together. If we didn't have to deal with the overlapping L1 > tables, we could compact a higher number of L0 sstables together into a set > of non-overlapping L1 sstables. > This could be done by delaying the invariant that L1 has no overlapping > sstables. Going from L1 to L2, we would be compacting fewer sstables together > which overlap. > When reading, we will not have the same one sstable per level (except L0) > guarantee, but this can be bounded (once we have too many sets of sstables, > either compact them back into the same level, or compact them up to the next > level). > This could be generalized to allow any level to be the maximum for this > overlapping strategy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7995) sstablerepairedset should take more that one sstable as an argument
[ https://issues.apache.org/jira/browse/CASSANDRA-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-7995: -- Component/s: Tools > sstablerepairedset should take more that one sstable as an argument > --- > > Key: CASSANDRA-7995 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7995 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Nick Bailey >Assignee: Carl Yeksigian > Labels: lhf > Fix For: 2.1.1 > > Attachments: 7995-2.1.txt > > > Given that a c* node can have a number of sstables in the 10s (100s?) of > thousands of sstables on it, sstablerepairedset should be taking a list of > sstables to mark as repaired rather than a single sstable. > Running any command 10s of thousands of times isn't really good let alone one > that spins up a jvm. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9499) Introduce writeVInt method to DataOutputStreamPlus
[ https://issues.apache.org/jira/browse/CASSANDRA-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9499: -- Component/s: Streaming and Messaging Local Write-Read Paths Compaction > Introduce writeVInt method to DataOutputStreamPlus > -- > > Key: CASSANDRA-9499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9499 > Project: Cassandra > Issue Type: Improvement > Components: Compaction, Local Write-Read Paths, Streaming and > Messaging >Reporter: Benedict >Assignee: Ariel Weisberg >Priority: Minor > Fix For: 3.0 alpha 1 > > > CASSANDRA-8099 really could do with a writeVInt method, for both fixing > CASSANDRA-9498 but also efficiently encoding timestamp/deletion deltas. It > should be possible to make an especially efficient implementation against > BufferedDataOutputStreamPlus. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9511) ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata fails on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-9511:
--
Component/s: Testing
> ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata fails on trunk
> ---
>
> Key: CASSANDRA-9511
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9511
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>
> See
> http://cassci.datastax.com/view/trunk/job/trunk_testall/121/testReport/org.apache.cassandra.db/ColumnFamilyStoreTest/testSliceByNamesCommandOldMetadata/
> {noformat}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:171)
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.rename(SSTableWriter.java:266)
> at
> org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:766)
> at
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetadata(ColumnFamilyStoreTest.java:1125)
> {noformat}
> I changed the logging a bit to track down the failed rename
> {noformat}
> DEBUG [MemtableFlushWriter:1] 2015-05-29 11:50:13,600 Renaming
> build/test/cassandra/data:0/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/tmp-la-4-big-TOC.txt
> to build
> /test/cassandra/data:0/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/la-4-big-TOC.txt
> DEBUG [main] 2015-05-29 11:50:13,612 Failed renaming
> build/test/cassandra/data:0/ColumnFamilyStoreTest1/Standard1-62503c50061a11e5a1d32fd3bb27a264/la-4-big-TOC.txt
> to build/test/cassan
> dra/data:0/ColumnFamilyStoreTest1/Standard1-62503c50061a11e5a1d32fd3bb27a264/la-11-big-TOC.txt
> {noformat}
> Looks like MemtaleFlushWriter is racing with the unit test thread to rename
> the file. This may be an actual bug since a similar race could occur when
> nodetool invokes CFS.loadNewSSTables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9500) SequentialWriter should extend BufferedDataOutputStreamPlus
[ https://issues.apache.org/jira/browse/CASSANDRA-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9500: -- Component/s: Streaming and Messaging Local Write-Read Paths Compaction > SequentialWriter should extend BufferedDataOutputStreamPlus > --- > > Key: CASSANDRA-9500 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9500 > Project: Cassandra > Issue Type: Improvement > Components: Compaction, Local Write-Read Paths, Streaming and > Messaging >Reporter: Benedict >Assignee: Ariel Weisberg >Priority: Minor > Fix For: 3.0 beta 1 > > > SequentialWriter is the last piece left not implementing > DataOutputStreamPlus. It should not be too tricky to extend it, and it should > result in improved write throughput. Further, it will permit it to exploit > the more efficient implementation of writeVInt being delivered in > CASSANDRA-9499 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7397) BatchlogManagerTest unit test failing
[
https://issues.apache.org/jira/browse/CASSANDRA-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-7397:
--
Component/s: Testing
> BatchlogManagerTest unit test failing
> -
>
> Key: CASSANDRA-7397
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7397
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
> Environment: cassci: Debian Wheezy amd64 / Oracle JDK 1.7_60 / 8G SSD
> my machine: Debian Jessie amd64 / Oracle JDK 1.7_60 / 16G SSD
>Reporter: Michael Shuler
>Assignee: Carl Yeksigian
> Labels: qa-resolved
> Fix For: 2.0.9, 2.1 rc2, 2.2.0 beta 1
>
> Attachments: 7397.patch, system.log.txt
>
>
> This test is passing on cassci, but has failed on other dev machines. This is
> from my machine on trunk (I'm checking other versions).
> {noformat}
> [junit] Testsuite: org.apache.cassandra.db.BatchlogManagerTest
> [junit] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 83.015 sec
> [junit]
> [junit] - Standard Output ---
> [junit] WARN 19:48:53 Changing /127.0.0.1's host ID from
> 7d5feaa0-f26a-11e3-9d93-ed61e821317c to 952657a0-f26a-11e3-9d93-ed61e821317c
> [junit] WARN 19:48:53 Changing /127.0.0.1's host ID from
> 7d5feaa0-f26a-11e3-9d93-ed61e821317c to 952657a0-f26a-11e3-9d93-ed61e821317c
> [junit] - ---
> [junit] Testcase:
> testReplay(org.apache.cassandra.db.BatchlogManagerTest): FAILED
> [junit] expected:<500> but was:<0>
> [junit] junit.framework.AssertionFailedError: expected:<500> but was:<0>
> [junit] at
> org.apache.cassandra.db.BatchlogManagerTest.testReplay(BatchlogManagerTest.java:91)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.db.BatchlogManagerTest FAILED
> {noformat}
> DEBUG test log attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9528) Improve log output from unit tests
[ https://issues.apache.org/jira/browse/CASSANDRA-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9528: -- Component/s: Testing > Improve log output from unit tests > -- > > Key: CASSANDRA-9528 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9528 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 alpha 1 > > > * Single log output file per suite > * stdout/stderr to the same log file with proper interleaving > * Don't interleave interactive output from unit tests run concurrently to the > console. Print everything about the test once the test has completed. > * Fetch and compress log files as part of artifacts collected by cassci -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9581) pig-tests spend time waiting on /dev/random for SecureRandom
[ https://issues.apache.org/jira/browse/CASSANDRA-9581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-9581: -- Component/s: Testing > pig-tests spend time waiting on /dev/random for SecureRandom > > > Key: CASSANDRA-9581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9581 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 2.2.1, 3.0 alpha 1 > > > We don't need secure random numbers (for unit tests) so waiting for entropy > doesn't make much sense. Luckily Java makes it easy to point to /dev/urandom > for entropy. It also transparently handles it correctly on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10362) Potential bugs in MV
[
https://issues.apache.org/jira/browse/CASSANDRA-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10362:
---
Component/s: Coordination
> Potential bugs in MV
>
>
> Key: CASSANDRA-10362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10362
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Sylvain Lebresne
>Assignee: Carl Yeksigian
> Fix For: 3.0.0 rc1
>
>
> While reviewing CASSANDRA-9664, I notice a few points in {{View.java}} that
> potentially look wrong to me. As those aren't introduced by CASSANDRA-9664,
> I'm opening this ticket separatly. The points in question are:
> * In {{updateAffectsView}}, I don't think the {{row.hasComplexDeletion()}}
> check is correct: if an update only deletes a given collection and that
> collection isn't in the view at all, then I don't think we should care about
> it. Besides, {{hasComplexDeletion}} is not all that efficient and if there is
> a complex deletion, the following loop on the row will catch it (but only if
> the column is interesting). So I think that check should just be removed.
> * Also in {{updateAffectsView}}, I'm surprised we don't check
> {{row.partition.primaryKeyLivenessInfo().isEmpty()}}: if we only insert
> values for the primary key columns, this should still always impact the view.
> * In {{createForDeletionInfo}}, it seems we assume that a live
> {{DeletionInfo}} has either ranges or a partition level deletion, but it can
> have both and it seems we'll only consider the ranges if that's the case,
> which is incorrect.
> It's possible there is a reasoning I'm missing behind how some or all of
> those point are currently work, and if so I apologize, but I think in that
> case said reasoning at least needs to be added as a comment. And if they do
> are problems, it would be nice to add tests for all these cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10382) nodetool info doesn't show the correct DC and RACK
[
https://issues.apache.org/jira/browse/CASSANDRA-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10382:
---
Component/s: (was: Configuration)
Tools
> nodetool info doesn't show the correct DC and RACK
> --
>
> Key: CASSANDRA-10382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10382
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Cassandra 2.2.1
> GossipingPropertyFileSnitch
>Reporter: Ruggero Marchei
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 2.1.10
>
>
> When running *nodetool info* cassandra returns UNKNOWN_DC and UNKNOWN_RACK:
> {code}
> # nodetool info
> ID : b94f9ca0-f886-4111-a471-02f295573f37
> Gossip active : true
> Thrift active : true
> Native Transport active: true
> Load : 44.97 MB
> Generation No : 1442913138
> Uptime (seconds) : 5386
> Heap Memory (MB) : 429.07 / 3972.00
> Off Heap Memory (MB) : 0.08
> Data Center: UNKNOWN_DC
> Rack : UNKNOWN_RACK
> Exceptions : 1
> Key Cache : entries 642, size 58.16 KB, capacity 100 MB, 5580
> hits, 8320 requests, 0.671 recent hit rate, 14400 save period in seconds
> Row Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> Counter Cache : entries 0, size 0 bytes, capacity 50 MB, 0 hits, 0
> requests, NaN recent hit rate, 7200 save period in seconds
> Token : (invoke with -T/--tokens to see all 256 tokens)
> {code}
> Correct DCs and RACKs are returned by *nodetool status* and *nodetool
> gossipinfo* commands:
> {code}
> # nodetool gossipinfo|grep -E 'RACK|DC'
> DC:POZ
> RACK:RACK30
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK68
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK62
> DC:SJC
> RACK:RACK62
> {code}
> {code}
> # nodetool status|grep Datacenter
> Datacenter: SJC
> Datacenter: POZ
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10177) materialized_views_test.TestMaterializedViews.really_complex_repair_test if flakey
[
https://issues.apache.org/jira/browse/CASSANDRA-10177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10177:
---
Component/s: Testing
> materialized_views_test.TestMaterializedViews.really_complex_repair_test if
> flakey
> --
>
> Key: CASSANDRA-10177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10177
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Sylvain Lebresne
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0 beta 2
>
>
> {{materialized_views_test.TestMaterializedViews.really_complex_repair_test}}
> if failing from times to times according to [its
> history|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/lastCompletedBuild/testReport/junit/materialized_views_test/TestMaterializedViews/really_complex_repair_test/history/].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10161) Composite case-sensitive primary key: first item is not quoted in DESCRIBE TABLE
[
https://issues.apache.org/jira/browse/CASSANDRA-10161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10161:
---
Component/s: Tools
> Composite case-sensitive primary key: first item is not quoted in DESCRIBE
> TABLE
>
>
> Key: CASSANDRA-10161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10161
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Julien Moreau
>Assignee: Carl Yeksigian
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.1.11
>
>
> A table is created with case-sensitive composite primary key:
> {code}
> CREATE TABLE foo (
> "Key1" text,
> "Key2" text,
> "Value" text
> PRIMARY KEY ("Key1", "Key2")
> );
> {code}
> In the result of {{DESCRIBE TABLE foo;}}:
> {code}
> CREATE TABLE foo (
> "Key1" text,
> "Key2" text,
> "Value" text
> PRIMARY KEY (Key1, "Key2")
> );
> {code}
> {{Key1}} is not quoted.
> When trying to re-create the table with this description, there is an error:
> {code}
> InvalidRequest: code=2200 [Invalid query] message="Unknown definition key1
> referenced in PRIMARY KEY"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10296) Aggregates aren't resolved properly for reversed types
[
https://issues.apache.org/jira/browse/CASSANDRA-10296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10296:
---
Component/s: CQL
> Aggregates aren't resolved properly for reversed types
> --
>
> Key: CASSANDRA-10296
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10296
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
> Fix For: 2.2.2
>
>
> When using an aggregate, it won't resolve if the column is reversed.
> {code}
> CREATE TABLE t1 (pk int, ck int, PRIMARY KEY (pk, ck)) WITH CLUSTERING ORDER
> BY (ck DESC)
> SELECT min(ck) FROM t1 WHERE pk = 1
> {code}
> results in the error:
> {noformat}
> Ambiguous call to function min (can be matched by following signatures:
> system.min : (varint) -> varint, system.min : (int) -> int, system.min :
> (blob) -> blob): use type casts to disambiguate
> {noformat}
> For aggregates, we should be treating a reversed type the same as we do a
> normal type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10382) nodetool info doesn't show the correct DC and RACK
[
https://issues.apache.org/jira/browse/CASSANDRA-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10382:
---
Component/s: Configuration
> nodetool info doesn't show the correct DC and RACK
> --
>
> Key: CASSANDRA-10382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10382
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
> Environment: Cassandra 2.2.1
> GossipingPropertyFileSnitch
>Reporter: Ruggero Marchei
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 2.1.10
>
>
> When running *nodetool info* cassandra returns UNKNOWN_DC and UNKNOWN_RACK:
> {code}
> # nodetool info
> ID : b94f9ca0-f886-4111-a471-02f295573f37
> Gossip active : true
> Thrift active : true
> Native Transport active: true
> Load : 44.97 MB
> Generation No : 1442913138
> Uptime (seconds) : 5386
> Heap Memory (MB) : 429.07 / 3972.00
> Off Heap Memory (MB) : 0.08
> Data Center: UNKNOWN_DC
> Rack : UNKNOWN_RACK
> Exceptions : 1
> Key Cache : entries 642, size 58.16 KB, capacity 100 MB, 5580
> hits, 8320 requests, 0.671 recent hit rate, 14400 save period in seconds
> Row Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> Counter Cache : entries 0, size 0 bytes, capacity 50 MB, 0 hits, 0
> requests, NaN recent hit rate, 7200 save period in seconds
> Token : (invoke with -T/--tokens to see all 256 tokens)
> {code}
> Correct DCs and RACKs are returned by *nodetool status* and *nodetool
> gossipinfo* commands:
> {code}
> # nodetool gossipinfo|grep -E 'RACK|DC'
> DC:POZ
> RACK:RACK30
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK68
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK62
> DC:SJC
> RACK:RACK62
> {code}
> {code}
> # nodetool status|grep Datacenter
> Datacenter: SJC
> Datacenter: POZ
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10242) Validate rack information on startup
[ https://issues.apache.org/jira/browse/CASSANDRA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-10242: --- Component/s: Configuration > Validate rack information on startup > > > Key: CASSANDRA-10242 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10242 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jonathan Ellis >Assignee: Carl Yeksigian > Fix For: 2.1.12, 2.2.4, 3.0.0 rc2 > > > Moving to a new rack means that different data should be stored on a node. > We already persist rack information in a system table; we should fail startup > if this doesn't match what the snitch thinks it should be. (Either the > snitch is wrong, and needs to be fixed, or the machine has been moved and > needs to be rebootstrapped.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10243) Warn or fail when changing cluster topology live
[
https://issues.apache.org/jira/browse/CASSANDRA-10243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023072#comment-15023072
]
Paulo Motta commented on CASSANDRA-10243:
-
Looking good, a few nits left:
* It's not very common to update snitch property of a live node (specially with
GPFS), so I don't think the warning is necessary. Should we maybe add a note to
NEWS.txt instead? Or maybe print the warn on 2.1/2.2, and remove it on 3.0?
* On CASSANDRA-9474 we changed the property name from {{override_rackdc}} to
the original {{ignore_rack}} in addition to {{ignore_dc}}, could you please fix
dtests?
* Instead of manipulating gossip {{EndpointState}} in
{{StorageProxy.truncateBlocking()}}, what do you think about adding a
excludeNodesInDeadState option to {{StorageService.getLiveMembers}} and
{{Gossiper.getLiveMembers}} (which you could also rename it to
{{Gossiper.getLiveEndpoints}})?
* Please fix
[nodetool_test.TestNodetool.test_correct_dc_rack_in_nodetool_info|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10243-2.2-dtest/lastCompletedBuild/testReport/nodetool_test/TestNodetool/test_correct_dc_rack_in_nodetool_info/]
dtest
* It seems [3.1
dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10243-3.1-dtest/]
did not run, mind resubmitting please?
Thanks!
> Warn or fail when changing cluster topology live
>
>
> Key: CASSANDRA-10243
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10243
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Jonathan Ellis
>Assignee: Stefania
>Priority: Critical
> Fix For: 2.1.x
>
>
> Moving a node from one rack to another in the snitch, while it is alive, is
> almost always the wrong thing to do.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10412) Could not initialize class org.apache.cassandra.config.DatabaseDescriptor
[ https://issues.apache.org/jira/browse/CASSANDRA-10412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-10412: --- Component/s: Configuration > Could not initialize class org.apache.cassandra.config.DatabaseDescriptor > - > > Key: CASSANDRA-10412 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10412 > Project: Cassandra > Issue Type: Bug > Components: Configuration > Environment: OS: Windows 2012 R2 > Java JRE: 1.8.0_51 >Reporter: Eric Simmons >Assignee: Carl Yeksigian >Priority: Minor > Fix For: 2.2.3, 3.0.0 rc2 > > Attachments: cassandra-env.ps1, cassandra.yaml > > > We are unable to start version 2.2.1 on our Windows 2012 R2 systems, however > we can use the same environment to start version 2.1.2 > I have attached our Cassandra.yaml and cassandra-env.ps1 file for reference. > I have also attached the system.log file displaying the error. > I have also included an excerpt of the log file showing the stack trace of > the error. > ERROR [ScheduledTasks:1] 2015-09-29 07:03:47,482 > DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor > java.lang.NoClassDefFoundError: Could not initialize class > org.apache.cassandra.config.DatabaseDescriptor > at > org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:57) > ~[apache-cassandra-2.2.1.jar:2.2.1] > at > org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:122) > ~[apache-cassandra-2.2.1.jar:2.2.1] > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) > ~[na:1.8.0_51] > at java.util.concurrent.FutureTask.runAndReset(Unknown Source) > ~[na:1.8.0_51] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown > Source) ~[na:1.8.0_51] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown > Source) ~[na:1.8.0_51] > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > [na:1.8.0_51] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > [na:1.8.0_51] > at java.lang.Thread.run(Unknown Source) [na:1.8.0_51] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10570) HSHA test_closing_connections test flapping on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-10570: --- Component/s: Testing > HSHA test_closing_connections test flapping on 3.0 > -- > > Key: CASSANDRA-10570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10570 > Project: Cassandra > Issue Type: Sub-task > Components: Testing >Reporter: Jim Witschey >Assignee: Carl Yeksigian > Fix For: 3.0.0 > > > See history here: > http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/280/testReport/thrift_hsha_test/ThriftHSHATest/test_closing_connections/history/ > The 3 most recent failures were: > http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/272/testReport/junit/thrift_hsha_test/ThriftHSHATest/test_closing_connections/ > http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/272/testReport/junit/thrift_hsha_test/ThriftHSHATest/test_closing_connections/ > http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/280/testReport/junit/thrift_hsha_test/ThriftHSHATest/test_closing_connections/ > I haven't seen this failure on CassCI on the 2.2 branch, so it may be a > regression; I'm not sure. > This is related to CASSANDRA-9369; it presents itself the same way. Looks > like that ticket was closed after a fix was merged to the dtests, but it > didn't fix this particular error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10457) fix failing jmx_metrics dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-10457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10457:
---
Component/s: Testing
> fix failing jmx_metrics dtest
> -
>
> Key: CASSANDRA-10457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10457
> Project: Cassandra
> Issue Type: Sub-task
> Components: Testing
>Reporter: Jim Witschey
>Assignee: Carl Yeksigian
> Fix For: 2.2.3, 3.0.0 rc2
>
>
> {{jmxmetrics_test.py:TestJMXMetrics.begin_test}} is failing on CassCI; I've
> reproduced it on OpenStack as well. Here's the failure:
> http://cassci.datastax.com/view/trunk/job/cassandra-3.0_dtest/lastCompletedBuild/testReport/jmxmetrics_test/TestJMXMetrics/begin_test/
> My first thought is that a fix may be as simple as changing the parameters to
> stress between collecting the before and after metrics. I haven't debugged
> this further than reproducing it, though.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9992) Sending batchlog verb to previous versions
[
https://issues.apache.org/jira/browse/CASSANDRA-9992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-9992:
--
Component/s: Coordination
> Sending batchlog verb to previous versions
> --
>
> Key: CASSANDRA-9992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9992
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
> Fix For: 3.0 beta 1
>
>
> We are currently sending {{Verb.BATCHLOG_MUTATION}} in
> {{StorageProxy.syncWriteToBatchlog}} and
> {{StorageProxy.asyncRemoveFromBatchlog}}. to previous versions which do not
> have that Verb. We should be sending them {{Verb.MUTATION}} instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10584) reads with EACH_QUORUM on keyspace with SimpleTopologyStrategy throw a ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-10584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10584:
---
Component/s: CQL
> reads with EACH_QUORUM on keyspace with SimpleTopologyStrategy throw a
> ClassCastException
> --
>
> Key: CASSANDRA-10584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10584
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Andy Tolbert
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 3.0.0
>
>
> I think this may be a regression introduced w/ [CASSANDRA-9602]. Starting
> with C* 3.0.0-rc2 an error is returned when querying a keyspace with
> {{SimpleTopologyStrategy}} using EACH_QUORUM CL:
> {noformat}
> cqlsh> create keyspace test with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> create table test.test (k int PRIMARY KEY, i int);
> cqlsh> consistency EACH_QUORUM;
> Consistency level set to EACH_QUORUM.
> cqlsh> select * from test.test;
> ServerError: message="java.lang.ClassCastException:
> org.apache.cassandra.locator.SimpleStrategy cannot be cast to
> org.apache.cassandra.locator.NetworkTopologyStrategy">
> {noformat}
> The exception yielded in the system logs:
> {noformat}
> ERROR [SharedPool-Worker-1] 2015-10-23 13:02:15,405 ErrorMessage.java:336 -
> Unexpected exception during request
> java.lang.ClassCastException: org.apache.cassandra.locator.SimpleStrategy
> cannot be cast to org.apache.cassandra.locator.NetworkTopologyStrategy
> at
> org.apache.cassandra.db.ConsistencyLevel.filterForEachQuorum(ConsistencyLevel.java:227)
> ~[main/:na]
> at
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:188)
> ~[main/:na]
> at
> org.apache.cassandra.db.ConsistencyLevel.filterForQuery(ConsistencyLevel.java:180)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1795)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$RangeIterator.computeNext(StorageProxy.java:1762)
> ~[main/:na]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[main/:na]
> at
> com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1149)
> ~[guava-18.0.jar:na]
> at
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1814)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$RangeMerger.computeNext(StorageProxy.java:1799)
> ~[main/:na]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1925)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$RangeCommandIterator.computeNext(StorageProxy.java:1892)
> ~[main/:na]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.WrappingPartitionIterator.hasNext(WrappingPartitionIterator.java:33)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.CountingPartitionIterator.hasNext(CountingPartitionIterator.java:49)
> ~[main/:na]
> at
> org.apache.cassandra.service.pager.AbstractQueryPager$PagerIterator.hasNext(AbstractQueryPager.java:99)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:610)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:371)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:327)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:213)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:205)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:236)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:221)
> ~[main/:na]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:11
[jira] [Updated] (CASSANDRA-10014) Deletions using clustering keys not reflected in MV
[
https://issues.apache.org/jira/browse/CASSANDRA-10014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-10014:
---
Component/s: Coordination
> Deletions using clustering keys not reflected in MV
> ---
>
> Key: CASSANDRA-10014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10014
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Stefan Podkowinski
>Assignee: Carl Yeksigian
> Fix For: 3.0 beta 1
>
>
> I wrote a test to reproduce an
> [issue|http://stackoverflow.com/questions/31810841/cassandra-materialized-view-shows-stale-data/31860487]
> reported on SO and turns out this is easily reproducible. There seems to be
> a bug preventing deletes to be propagated to MVs in case a clustering key is
> used. See
> [here|https://github.com/spodkowinski/cassandra/commit/1c064523c8d8dbee30d46a03a0f58d3be97800dc]
> for test case (testClusteringKeyTombstone should fail).
> It seems {{MaterializedView.updateAffectsView()}} will not consider the
> delete relevant for the view as {{partition.deletionInfo().isLive()}} will be
> true during the test. In other test cases isLive will return false, which
> seems to be the actual problem here. I'm not even sure the root cause is MV
> specific, but wasn't able to dig much deeper as I'm not familiar with the
> slightly confusing semantics around DeletionInfo.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9921) Combine MV schema definition with MV table definition
[
https://issues.apache.org/jira/browse/CASSANDRA-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Yeksigian updated CASSANDRA-9921:
--
Component/s: Distributed Metadata
> Combine MV schema definition with MV table definition
> -
>
> Key: CASSANDRA-9921
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9921
> Project: Cassandra
> Issue Type: Improvement
> Components: Distributed Metadata
>Reporter: Carl Yeksigian
>Assignee: Carl Yeksigian
> Labels: client-impacting, materializedviews
> Fix For: 3.0.0 rc1
>
> Attachments: 9921-unit-test.txt
>
>
> Prevent MV from reusing {{system_schema.tables}} and instead move those
> properties into the {{system_schema.materializedviews}} table to keep them
> separate entities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
