[jira] [Created] (CASSANDRA-8683) Incremental repairs broken with early opening of compaction results
Marcus Eriksson created CASSANDRA-8683: -- Summary: Incremental repairs broken with early opening of compaction results Key: CASSANDRA-8683 URL: https://issues.apache.org/jira/browse/CASSANDRA-8683 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.1.3 Incremental repairs holds a set of the sstables it started the repair on (we need to know which sstables were actually validated to be able to anticompact them). This includes any tmplink files that existed when the compaction started (if we wouldn't include those, we would miss data since we move the start point of the existing non-tmplink files) With CASSANDRA-6916 we swap out those instances with new ones (SSTR.cloneWithNewStart / SSTW.openEarly), meaning that the underlying file can get deleted even though we hold a reference. This causes the unit test error: http://cassci.datastax.com/job/trunk_utest/1330/testReport/junit/org.apache.cassandra.db.compaction/LeveledCompactionStrategyTest/testValidationMultipleSSTablePerLevel/ (note that it only fails on trunk though, in 2.1 we don't hold references to the repairing files for non-incremental repairs, but the bug should exist in 2.1 as well) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8677) rpc_interface and listen_interface generate NPE on startup when specified interface doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291843#comment-14291843
]
Robert Stupp commented on CASSANDRA-8677:
-
Mostly LGTM - some minors I'd like to have included in the patch(es):
* would love to see a {{hasMoreElements()}} check before {{*Address =
addrs.nextElement()}} and throw an appropriate exception
* maybe merge the code {{listen_address}} and {{rpc_address}} into a single
method - it looks very similar. E.g. {{private static InetAddress
networkInterfaceFromConfiguration(String confValue, String confField) {\\} }}
rpc_interface and listen_interface generate NPE on startup when specified
interface doesn't exist
-
Key: CASSANDRA-8677
URL: https://issues.apache.org/jira/browse/CASSANDRA-8677
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Attachments: 8677-2.1.patch, 8677.patch
This is just a buggy UI bit.
Initially the error I got was this which is redundant and not well formatted.
{noformat}
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
Exception encountered during startup: null
Fatal configuration error; unable to start. See log for stacktrace.
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
This has no description of the error that occurred. After logging the
exception.
{noformat}
java.lang.NullPointerException: null
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:347)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:102)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
Exceptions thrown in the DatabaseDescriptor should log in a useful way.
This particular error should generate a message without a stack trace since
it is easily recognized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8677) rpc_interface and listen_interface generate NPE on startup when specified interface doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291843#comment-14291843
]
Robert Stupp edited comment on CASSANDRA-8677 at 1/26/15 2:07 PM:
--
Mostly LGTM - some minors I'd like to have included in the patch(es):
* would love to see a {{hasMoreElements()}} check before {{*Address =
addrs.nextElement()}} and throw an appropriate exception
* maybe merge the code {{listen_address}} and {{rpc_address}} into a single
method - it looks very similar. E.g. {{private static InetAddress
networkInterfaceFromConfiguration(String confValue, String confField)}}
was (Author: snazy):
Mostly LGTM - some minors I'd like to have included in the patch(es):
* would love to see a {{hasMoreElements()}} check before {{*Address =
addrs.nextElement()}} and throw an appropriate exception
* maybe merge the code {{listen_address}} and {{rpc_address}} into a single
method - it looks very similar. E.g. {{private static InetAddress
networkInterfaceFromConfiguration(String confValue, String confField) ... }}
rpc_interface and listen_interface generate NPE on startup when specified
interface doesn't exist
-
Key: CASSANDRA-8677
URL: https://issues.apache.org/jira/browse/CASSANDRA-8677
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Attachments: 8677-2.1.patch, 8677.patch
This is just a buggy UI bit.
Initially the error I got was this which is redundant and not well formatted.
{noformat}
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
Exception encountered during startup: null
Fatal configuration error; unable to start. See log for stacktrace.
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
This has no description of the error that occurred. After logging the
exception.
{noformat}
java.lang.NullPointerException: null
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:347)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:102)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
Exceptions thrown in the DatabaseDescriptor should log in a useful way.
This particular error should generate a message without a stack trace since
it is easily recognized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8677) rpc_interface and listen_interface generate NPE on startup when specified interface doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291843#comment-14291843
]
Robert Stupp edited comment on CASSANDRA-8677 at 1/26/15 2:06 PM:
--
Mostly LGTM - some minors I'd like to have included in the patch(es):
* would love to see a {{hasMoreElements()}} check before {{*Address =
addrs.nextElement()}} and throw an appropriate exception
* maybe merge the code {{listen_address}} and {{rpc_address}} into a single
method - it looks very similar. E.g. {{private static InetAddress
networkInterfaceFromConfiguration(String confValue, String confField) ... }}
was (Author: snazy):
Mostly LGTM - some minors I'd like to have included in the patch(es):
* would love to see a {{hasMoreElements()}} check before {{*Address =
addrs.nextElement()}} and throw an appropriate exception
* maybe merge the code {{listen_address}} and {{rpc_address}} into a single
method - it looks very similar. E.g. {{private static InetAddress
networkInterfaceFromConfiguration(String confValue, String confField) {\\} }}
rpc_interface and listen_interface generate NPE on startup when specified
interface doesn't exist
-
Key: CASSANDRA-8677
URL: https://issues.apache.org/jira/browse/CASSANDRA-8677
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Attachments: 8677-2.1.patch, 8677.patch
This is just a buggy UI bit.
Initially the error I got was this which is redundant and not well formatted.
{noformat}
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
Exception encountered during startup: null
Fatal configuration error; unable to start. See log for stacktrace.
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
This has no description of the error that occurred. After logging the
exception.
{noformat}
java.lang.NullPointerException: null
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:347)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:102)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
Exceptions thrown in the DatabaseDescriptor should log in a useful way.
This particular error should generate a message without a stack trace since
it is easily recognized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8682) BulkRecordWriter ends up streaming with non-unique session IDs on large hadoop cluster
Erik Forsberg created CASSANDRA-8682:
Summary: BulkRecordWriter ends up streaming with non-unique
session IDs on large hadoop cluster
Key: CASSANDRA-8682
URL: https://issues.apache.org/jira/browse/CASSANDRA-8682
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Reporter: Erik Forsberg
Attachments: cassandra-1.2-bulkrecordwriter-sessionid.patch
We use BulkOutputFormat extensively to load data from hadoop to Cassandra. We
are currently running Cassandra 1.2.18, but are planning an upgrade of
Cassandra to 2.0.X, possibly 2.1.X.
With Cassandra 1.2 we have problems with the streaming session IDs getting
duplicated when multiple (20+) java processes start to do streaming at the same
time. On the receiving cassandra node, having the same session ID actually
correspond to different sending processing would confuse things a lot, leading
to aborted connections.
This would not happen for every process, but often enough to be a problem in
production environment. So it was a bit tricky to test.
Suspecting this have to do with how UUIDs are generated on the sending (hadoop
side). With 20+ processes being started concurrently, the clockSeqAndNode part
of the uuid1 probably ended up being exactly the same on all 20 processes.
I wrote a patch which I unfortunately never submitted at the time, but it's
attached to this issue. The patch constructs a UUID from the map or reduce task
ID, which is guaranteed to be unique per hadoop cluster.
I suspect we're going to face the same issue on Cassandra 2.0 and 2.1, even
after the rewrite of the streaming subsystem. Please correct me if I'm wrong,
i.e. if there's something in the new code that will make this a non-issue.
Now the question is how to address this problem. Possible options that I see
after some code reading:
1. Update patch to apply on 2.0 and 2.1, using same method (generating UUID
from hadoop task ID)
2. Modify UUIDGen code to use java process pid as clockSeq instead of random
number. However, getting the pid in java seems less than simple (and remember
that this is code that runs on the hadoop size of things, not inside cassandra
daemon)
3. This patch might help:
{noformat}
diff --git a/src/java/org/apache/cassandra/utils/UUIDGen.java
b/src/java/org/apache/cassandra/utils/UUIDGen.java
index f385744..ae253ab 100644
--- a/src/java/org/apache/cassandra/utils/UUIDGen.java
+++ b/src/java/org/apache/cassandra/utils/UUIDGen.java
@@ -234,7 +234,7 @@ public class UUIDGen
private static long makeClockSeqAndNode()
{
-long clock = new Random(System.currentTimeMillis()).nextLong();
+long clock = new Random().nextLong();
long lsb = 0;
lsb |= 0x8000L; // variant (2 bits)
{noformat}
..but I don't know the reason System.currentTimeMillis() is being used.
Opinions?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8683) Incremental repairs broken with early opening of compaction results
[ https://issues.apache.org/jira/browse/CASSANDRA-8683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291873#comment-14291873 ] Benedict commented on CASSANDRA-8683: - I'm a little confused by the description: if it holds a reference, it shouldn't delete the file. Is it possible it's releasing its reference early somehow? Incremental repairs broken with early opening of compaction results --- Key: CASSANDRA-8683 URL: https://issues.apache.org/jira/browse/CASSANDRA-8683 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.1.3 Incremental repairs holds a set of the sstables it started the repair on (we need to know which sstables were actually validated to be able to anticompact them). This includes any tmplink files that existed when the compaction started (if we wouldn't include those, we would miss data since we move the start point of the existing non-tmplink files) With CASSANDRA-6916 we swap out those instances with new ones (SSTR.cloneWithNewStart / SSTW.openEarly), meaning that the underlying file can get deleted even though we hold a reference. This causes the unit test error: http://cassci.datastax.com/job/trunk_utest/1330/testReport/junit/org.apache.cassandra.db.compaction/LeveledCompactionStrategyTest/testValidationMultipleSSTablePerLevel/ (note that it only fails on trunk though, in 2.1 we don't hold references to the repairing files for non-incremental repairs, but the bug should exist in 2.1 as well) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8613) Regression in mixed single and multi-column relation support
[
https://issues.apache.org/jira/browse/CASSANDRA-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291884#comment-14291884
]
Jared Gray commented on CASSANDRA-8613:
---
I originally worked with Hobbs to open this issue. I've linked to a design
document below, describing our internal use of multi-column relations in
Cassandra. Though all of these queries worked in version 2.0.6, many have
broken since Cassandra 2.0.9.
[Paging and Filtering with Apache
Cassandra|https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Wf96854c0c8fc_4762_9b83_c6247feca5fc/page/Paging%20and%20Filtering%20with%20Apache%20Cassandra]
Regression in mixed single and multi-column relation support
Key: CASSANDRA-8613
URL: https://issues.apache.org/jira/browse/CASSANDRA-8613
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Tyler Hobbs
Assignee: Benjamin Lerer
Fix For: 2.1.3, 2.0.13
In 2.0.6 through 2.0.8, a query like the following was supported:
{noformat}
SELECT * FROM mytable WHERE clustering_0 = ? AND (clustering_1, clustering_2)
(?, ?)
{noformat}
However, after CASSANDRA-6875, you'll get the following error:
{noformat}
Clustering columns may not be skipped in multi-column relations. They should
appear in the PRIMARY KEY order. Got (c, d) (0, 0)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8683) Incremental repairs broken with early opening of compaction results
[
https://issues.apache.org/jira/browse/CASSANDRA-8683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291895#comment-14291895
]
Marcus Eriksson commented on CASSANDRA-8683:
from irc:
{noformat}
15:27 marcuse belliottsmith: we create a new sstable reader over the same
file, the reference counter is not transfered to the new file, so if i bump the
reference count on the file i hold a reference to, it will not affect the
latest one
15:27 marcuse does it make sense/
15:27 marcuse * transfered to the new SSTR instance
15:28 belliottsmith but you should have taken a reference against the one
you're using, no?
15:28 marcuse belliottsmith: no, we don't want to hold references during the
whole duration of the repair (hours)
15:28 belliottsmith each instance is distinct, but so long as you take a
reference against the one you're using, it shouldn't be a problem...
15:28 marcuse so, if a file is gone, we simply remove it from the set and
dont anticompact it
15:29 belliottsmith ah
15:29 belliottsmith hmm. that is tricky, since we need to repeatedly refetch
the set of files we want to use, since we could have new data replacing it as
well, surely?
15:30 belliottsmith i don't really see a way around holding a reference for
the lifetime of the repair. but if we stream oldest files first we can drop
references to them as we go, no?
15:30 marcuse no, for incremental repairs we repair best-effort, if a file
is gone, it means it has been compacted away and will get repaired next time
around
15:31 belliottsmith so where/when do we grab our reference to the sstable
then?
15:31 belliottsmith because in that case it sounds like we should simply be
failing to grab our reference count
15:31 marcuse
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/ActiveRepairService.java#L409
15:32 marcuse and yes, we probably should fail, but we don't
15:32 belliottsmith that code looks a bit odd. why do we check for data
component first? shouldn't an attempt to acquire the ref be sufficient?
(clearly it isn't here though, since it's not removing it from the set)
15:33 marcuse yes, it should, but i don't think we decrease ref count when
we replace an instance?
15:34 marcuse ie, when we open early and have a new instance, we should
decrease the reference on the old file
15:34 marcuse *old instance\
15:34 belliottsmith we have to
15:34 belliottsmith otherwise it would never get deleted
15:34 marcuse but its the same underlying file right? it will get deleted by
the new instance
15:34 belliottsmith if we don't that's definitely a bug, but i'm pretty sure
we do
15:34 belliottsmith but lifetimes don't expire in order necessarily
15:35 belliottsmith so the older files ref count to zero, then remove
themselves from the linked-list of replacements
15:35 belliottsmith if there's nobody left, they delete the file
15:36 belliottsmith this all is in dire need of cleaning up in 8568
{noformat}
Incremental repairs broken with early opening of compaction results
---
Key: CASSANDRA-8683
URL: https://issues.apache.org/jira/browse/CASSANDRA-8683
Project: Cassandra
Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
Fix For: 2.1.3
Incremental repairs holds a set of the sstables it started the repair on (we
need to know which sstables were actually validated to be able to anticompact
them). This includes any tmplink files that existed when the compaction
started (if we wouldn't include those, we would miss data since we move the
start point of the existing non-tmplink files)
With CASSANDRA-6916 we swap out those instances with new ones
(SSTR.cloneWithNewStart / SSTW.openEarly), meaning that the underlying file
can get deleted even though we hold a reference.
This causes the unit test error:
http://cassci.datastax.com/job/trunk_utest/1330/testReport/junit/org.apache.cassandra.db.compaction/LeveledCompactionStrategyTest/testValidationMultipleSSTablePerLevel/
(note that it only fails on trunk though, in 2.1 we don't hold references to
the repairing files for non-incremental repairs, but the bug should exist in
2.1 as well)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8574) Gracefully degrade SELECT when there are lots of tombstones
[ https://issues.apache.org/jira/browse/CASSANDRA-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291814#comment-14291814 ] Brandon Williams commented on CASSANDRA-8574: - Yeah, I think it would at least have to return the last tombstone or row encountered, and somehow indicate if it was returning a result, or merely a tombstone. Gracefully degrade SELECT when there are lots of tombstones --- Key: CASSANDRA-8574 URL: https://issues.apache.org/jira/browse/CASSANDRA-8574 Project: Cassandra Issue Type: Improvement Reporter: Jens Rantil Fix For: 3.0 *Background:* There's lots of tooling out there to do BigData analysis on Cassandra clusters. Examples are Spark and Hadoop, which is offered by DSE. The problem with both of these so far, is that a single partition key with too many tombstones can make the query job fail hard. The described scenario happens despite the user setting a rather small FetchSize. I assume this is a common scenario if you have larger rows. *Proposal:* To allow a CQL SELECT to gracefully degrade to only return a smaller batch of results if there are too many tombstones. The tombstones are ordered according to clustering key and one should be able to page through them. Potentially: SELECT * FROM mytable LIMIT 1000 TOMBSTONES; would page through maximum 1000 tombstones, _or_ 1000 (CQL) rows. I understand that this obviously would degrade performance, but it would at least yield a result. *Additional comment:* I haven't dug into Cassandra code, but conceptually I guess this would be doable. Let me know what you think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
Ariel Weisberg created CASSANDRA-8684: - Summary: Replace usage of Adler32 with CRC32 Key: CASSANDRA-8684 URL: https://issues.apache.org/jira/browse/CASSANDRA-8684 Project: Cassandra Issue Type: Bug Components: Core Reporter: Ariel Weisberg Attachments: CRCBenchmark.java, PureJavaCrc32.java I could not find a situation in which Adler32 outperformed PureJavaCrc32 much less the intrinsic from Java 8. For small allocations PureJavaCrc32 was much faster probably due to the JNI overhead of invoking the native Adler32 implementation where the array has to be allocated and copied. I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as well as a c3.8xlarge running Ubuntu 14.04. I think it makes sense to stop using Adler32 when generating new checksums. c3.8xlarge, results are time in milliseconds, lower is better ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|47636|46075|25782| |128|36755|36712|23782| |256|31194|32211|22731| |1024|27194|28792|22010| |1048576|25941|27807|21808| |536870912|25957|27840|21836| i5 ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|50539|50466|26826| |128|37092|38533|24553| |256|30630|32938|23459| |1024|26064|29079|22592| |1048576|24357|27911|22481| |536870912|24838|28360|22853| Another fun fact. Performance of the CRC32 intrinsic appears to double from Sandy Bridge - Haswell. Unless I am measuring something different when going from Linux/Sandy to Haswell/OS X. The intrinsic/JDK 8 implementation also operates against DirectByteBuffers better and coding against the wrapper will get that boost when run with Java 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8685) Consider upgrade to thrift 0.9.2 (or later)
Adam Hattrell created CASSANDRA-8685: Summary: Consider upgrade to thrift 0.9.2 (or later) Key: CASSANDRA-8685 URL: https://issues.apache.org/jira/browse/CASSANDRA-8685 Project: Cassandra Issue Type: Bug Reporter: Adam Hattrell Folks using Astyanax and the like are subject to https://issues.apache.org/jira/browse/THRIFT-1457 and may run into heap pressure on the Cassandra side for larger read request, as thrift doesn't reset its internal buffer. This can lead to larger TFramedTransport instances will be kept on the heap. I've seen at least one situation where this has saved around 1Gb of heap space on average. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8683) Incremental repairs broken with early opening of compaction results
[ https://issues.apache.org/jira/browse/CASSANDRA-8683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291991#comment-14291991 ] Alan Boudreault commented on CASSANDRA-8683: related to CASSANDRA-8366 ? Incremental repairs broken with early opening of compaction results --- Key: CASSANDRA-8683 URL: https://issues.apache.org/jira/browse/CASSANDRA-8683 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.1.3 Incremental repairs holds a set of the sstables it started the repair on (we need to know which sstables were actually validated to be able to anticompact them). This includes any tmplink files that existed when the compaction started (if we wouldn't include those, we would miss data since we move the start point of the existing non-tmplink files) With CASSANDRA-6916 we swap out those instances with new ones (SSTR.cloneWithNewStart / SSTW.openEarly), meaning that the underlying file can get deleted even though we hold a reference. This causes the unit test error: http://cassci.datastax.com/job/trunk_utest/1330/testReport/junit/org.apache.cassandra.db.compaction/LeveledCompactionStrategyTest/testValidationMultipleSSTablePerLevel/ (note that it only fails on trunk though, in 2.1 we don't hold references to the repairing files for non-incremental repairs, but the bug should exist in 2.1 as well) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8613) Regression in mixed single and multi-column relation support
[
https://issues.apache.org/jira/browse/CASSANDRA-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288158#comment-14288158
]
Benjamin Lerer edited comment on CASSANDRA-8613 at 1/26/15 4:29 PM:
The reason, I think, why it is not cover is that we never intended to support
this type of query. It is a side effect of how multi-column relations were
originally developed. In the following versions we implemented some checks to
prevent the mix of single column and multi-column relations on the clustering
keys. The refactoring of {{SelectStatement}} in CASSANDRA-7981 has enforced
even more that constraint in the code.
Overall, I agree with [~thobbs] that this query make sense and that we should
support it. It is just not an easy thing to do on the current code.
was (Author: blerer):
The reason, I think, while it is not cover is that we never intended to support
this type of query. It is a side effect of how multi-column relations were
originally developed. In the following versions we implemented some check to
prevent the mix of single column and multi column relation on the clustering
keys. The refactoring of {{SelectStatement}} in CASSANDRA-7981 has enforced
even more that constraint in the code.
Overall, I agree with [~thobbs] that this query make sense and that we should
support it. It is just not an easy thing to do on the current code.
Regression in mixed single and multi-column relation support
Key: CASSANDRA-8613
URL: https://issues.apache.org/jira/browse/CASSANDRA-8613
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Tyler Hobbs
Assignee: Benjamin Lerer
Fix For: 2.1.3, 2.0.13
In 2.0.6 through 2.0.8, a query like the following was supported:
{noformat}
SELECT * FROM mytable WHERE clustering_0 = ? AND (clustering_1, clustering_2)
(?, ?)
{noformat}
However, after CASSANDRA-6875, you'll get the following error:
{noformat}
Clustering columns may not be skipped in multi-column relations. They should
appear in the PRIMARY KEY order. Got (c, d) (0, 0)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8613) Regression in mixed single and multi-column relation support
[
https://issues.apache.org/jira/browse/CASSANDRA-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292043#comment-14292043
]
Benjamin Lerer commented on CASSANDRA-8613:
---
[~jgray] Thanks for the link.
I understand your problem. It is a perfectly valid issue and I will fix the
regression.
In my previous comment, I was just trying to explain why we did not have some
tests for this type of query and the problems behind this issue.
Regression in mixed single and multi-column relation support
Key: CASSANDRA-8613
URL: https://issues.apache.org/jira/browse/CASSANDRA-8613
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Tyler Hobbs
Assignee: Benjamin Lerer
Fix For: 2.1.3, 2.0.13
In 2.0.6 through 2.0.8, a query like the following was supported:
{noformat}
SELECT * FROM mytable WHERE clustering_0 = ? AND (clustering_1, clustering_2)
(?, ?)
{noformat}
However, after CASSANDRA-6875, you'll get the following error:
{noformat}
Clustering columns may not be skipped in multi-column relations. They should
appear in the PRIMARY KEY order. Got (c, d) (0, 0)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8686) Introduce Latency Target for Stress
jonathan lacefield created CASSANDRA-8686: - Summary: Introduce Latency Target for Stress Key: CASSANDRA-8686 URL: https://issues.apache.org/jira/browse/CASSANDRA-8686 Project: Cassandra Issue Type: Improvement Reporter: jonathan lacefield Priority: Minor This item is a request to add a latency target to the rate option for the new stress tool. The goal of the latency target would be to provide a guideline for SLAs to the stress tool so the stress tool can determine threads and throughputs that can be sustained while meeting the SLA targets. For example: cassandra-stress [options/commands] -rate latency p90=5 p95=10 p99=100 The outcome of this command would be a stress execution that would gradually increase threads, and hence throughput (trans/sec), until the latency profile can no longer be satisfied with the current workload (yaml file definition) and/or cluster. This would provide a ceiling for throughput and connections for the given cluster, workload, and SLA profile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
[ https://issues.apache.org/jira/browse/CASSANDRA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292333#comment-14292333 ] Brandon Williams commented on CASSANDRA-8687: - Note that whoever commits this should merge -s ours when merging to trunk since client mode does not exist there. Keyspace should also check Config.isClientMode -- Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Priority: Minor Fix For: 2.1.3, 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
[ https://issues.apache.org/jira/browse/CASSANDRA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-8687: --- Comment: was deleted (was: Patch for trunk) Keyspace should also check Config.isClientMode -- Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Priority: Minor Fix For: 2.1.3, 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8651) Add support for running on Apache Mesos
[ https://issues.apache.org/jira/browse/CASSANDRA-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291805#comment-14291805 ] Sylvain Lebresne commented on CASSANDRA-8651: - bq. There's also a question of whether or not it makes sense to have the executor code live in the Cassandra tree If it's developped and maintained by the folks at Mesosphere (and only useful to Mesos users), then I think it makes more sense to have it live in the Mesos tree. But I'm happy to defer the discussion to when said executor exists. Add support for running on Apache Mesos --- Key: CASSANDRA-8651 URL: https://issues.apache.org/jira/browse/CASSANDRA-8651 Project: Cassandra Issue Type: Task Reporter: Ben Whitehead Assignee: Albert P Tobey Priority: Minor Fix For: 3.0 As a user of Apache Mesos I would like to be able to run Cassandra on my Mesos cluster. This would entail integration of Cassandra on Mesos through the creation of a production level Mesos framework. This would enable me to avoid static partitioning and inefficiencies and run Cassandra as part of my data center infrastructure. http://mesos.apache.org/documentation/latest/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8683) Incremental repairs broken with early opening of compaction results
[ https://issues.apache.org/jira/browse/CASSANDRA-8683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291865#comment-14291865 ] Marcus Eriksson commented on CASSANDRA-8683: cc [~benedict] Incremental repairs broken with early opening of compaction results --- Key: CASSANDRA-8683 URL: https://issues.apache.org/jira/browse/CASSANDRA-8683 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.1.3 Incremental repairs holds a set of the sstables it started the repair on (we need to know which sstables were actually validated to be able to anticompact them). This includes any tmplink files that existed when the compaction started (if we wouldn't include those, we would miss data since we move the start point of the existing non-tmplink files) With CASSANDRA-6916 we swap out those instances with new ones (SSTR.cloneWithNewStart / SSTW.openEarly), meaning that the underlying file can get deleted even though we hold a reference. This causes the unit test error: http://cassci.datastax.com/job/trunk_utest/1330/testReport/junit/org.apache.cassandra.db.compaction/LeveledCompactionStrategyTest/testValidationMultipleSSTablePerLevel/ (note that it only fails on trunk though, in 2.1 we don't hold references to the repairing files for non-incremental repairs, but the bug should exist in 2.1 as well) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8268) Token serialization should not assume all tokens are created by the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292215#comment-14292215 ] Aleksey Yeschenko commented on CASSANDRA-8268: -- I'm assuming that this is stuck for lack of a reviewer. If so, I'll take it. Can you rebase for latest trunk? Token serialization should not assume all tokens are created by the database partitioner Key: CASSANDRA-8268 URL: https://issues.apache.org/jira/browse/CASSANDRA-8268 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Assignee: Branimir Lambov Priority: Minor Attachments: 8268.patch This is also a hidden reference to StorageService.getPartitioner within Token, but this needs to be handled more carefully as we need to check whether tokens serialized with the wrong partitioner could be present in user data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8268) Token serialization should not assume all tokens are created by the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8268: - Reviewer: Aleksey Yeschenko Token serialization should not assume all tokens are created by the database partitioner Key: CASSANDRA-8268 URL: https://issues.apache.org/jira/browse/CASSANDRA-8268 Project: Cassandra Issue Type: Bug Reporter: Branimir Lambov Assignee: Branimir Lambov Priority: Minor Attachments: 8268.patch This is also a hidden reference to StorageService.getPartitioner within Token, but this needs to be handled more carefully as we need to check whether tokens serialized with the wrong partitioner could be present in user data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8623) sstablesplit fails *randomly* with Data component is missing
[
https://issues.apache.org/jira/browse/CASSANDRA-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Boudreault updated CASSANDRA-8623:
---
Attachment: output2.log
[~krummas] Tried the patch. I'm now getting this exception:
{code}
ERROR 18:23:48 Error in ThreadPoolExecutor
java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
down
at
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
~[na:1.7.0_72]
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
~[na:1.7.0_72]
at
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:150)
~[main/:na]
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
~[na:1.7.0_72]
at
org.apache.cassandra.db.compaction.CompactionManager.submitBackground(CompactionManager.java:183)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:239)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_72]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_72]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_72]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_72]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
{code}
See output2.log for more information.
sstablesplit fails *randomly* with Data component is missing
Key: CASSANDRA-8623
URL: https://issues.apache.org/jira/browse/CASSANDRA-8623
Project: Cassandra
Issue Type: Bug
Reporter: Alan Boudreault
Assignee: Marcus Eriksson
Attachments:
0001-make-sure-we-finish-compactions-before-waiting-for-d.patch, output.log,
output2.log
I'm experiencing an issue related to sstablesplit. I would like to understand
if I am doing something wrong or there is an issue in the split process. The
process fails randomly with the following exception:
{code}
ERROR 02:17:36 Error in ThreadPoolExecutor
java.lang.AssertionError: Data component is missing for
sstable./tools/bin/../../data/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-16
{code}
See attached output.log file. The process never stops after this exception
and I've also seen the dataset growing indefinitely (number of sstables).
* I have not been able to reproduce the issue with a single sstablesplit
command. ie, specifying all files with glob matching.
* I can reproduce the bug if I call multiple sstablesplit one file at the
time (the way ccm does)
Here is the test case file to reproduce the bug:
https://drive.google.com/file/d/0BwZ_GPM33j6KdVh0NTdkOWV2R1E/view?usp=sharing
1. Download the split_issue.tar.gz file. It includes latest cassandra-2.1
branch binaries.
2. Extract it
3. CD inside the use case directory
4. Download the dataset (2G) just to be sure we have the same thing, and
place it in the working directory.
https://docs.google.com/uc?id=0BwZ_GPM33j6KV3ViNnpPcVFndUUexport=download
5. The first time, run ./test.sh. This will setup and run a test.
6. The next times, you can only run ./test --no-setup . This will only reset
the dataset as its initial state and re-run the test. You might have to run
the tests some times before experiencing it... but I'm always able with only
2-3 runs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
[ https://issues.apache.org/jira/browse/CASSANDRA-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg reassigned CASSANDRA-8684: - Assignee: Ariel Weisberg Replace usage of Adler32 with CRC32 --- Key: CASSANDRA-8684 URL: https://issues.apache.org/jira/browse/CASSANDRA-8684 Project: Cassandra Issue Type: Bug Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Attachments: CRCBenchmark.java, PureJavaCrc32.java I could not find a situation in which Adler32 outperformed PureJavaCrc32 much less the intrinsic from Java 8. For small allocations PureJavaCrc32 was much faster probably due to the JNI overhead of invoking the native Adler32 implementation where the array has to be allocated and copied. I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as well as a c3.8xlarge running Ubuntu 14.04. I think it makes sense to stop using Adler32 when generating new checksums. c3.8xlarge, results are time in milliseconds, lower is better ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|47636|46075|25782| |128|36755|36712|23782| |256|31194|32211|22731| |1024|27194|28792|22010| |1048576|25941|27807|21808| |536870912|25957|27840|21836| i5 ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|50539|50466|26826| |128|37092|38533|24553| |256|30630|32938|23459| |1024|26064|29079|22592| |1048576|24357|27911|22481| |536870912|24838|28360|22853| Another fun fact. Performance of the CRC32 intrinsic appears to double from Sandy Bridge - Haswell. Unless I am measuring something different when going from Linux/Sandy to Haswell/OS X. The intrinsic/JDK 8 implementation also operates against DirectByteBuffers better and coding against the wrapper will get that boost when run with Java 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292325#comment-14292325 ] Russ Hatch commented on CASSANDRA-8072: --- [~respringer] Can you help me better understand what's happening with the nodes (via opscenter) when this occurs? From what we talked about before it sounds like the nodes are started with little/no configuration and then are stopped, configured, and started up again with seeds configured. First startup: - are any changes made to cassandra.yaml before this start? - are nodes aware of each other as seeds? - does the start happen serially or in parallel? - is it possible to get sample config yaml for 2 nodes at this phase? Stopping: - are the nodes stopped serially or in parallel? - are the nodes able to complete startup before being stopped, or could they be getting interrupted during initial start? - are the nodes stopped forcefully (like kill -9) or something nicer? Starting again with configurations completed: - are nodes started serially or in parallel? (from what I know this would be parallel but just want to be sure) - will this startup step wait for all nodes to be ready before launching in parallel? (or could one node get a significant head start if it completes the earlier steps first?) - is it possible to get sample config yaml for 2 nodes at this point? (I grabbed yaml from a failed repro attempt but want to be sure I didn't get something wrong in that attempt) Finally, when provisioning in this way how do the (ec2) nodes refer to one another: public ip, private ip, or private dns? Thanks! And sorry for all the questions. I'm trying to close in on the issue and still having difficulty reproducing in a local container environment, so I'm trying to figure out what could be unique about the provisioning of these nodes that may account for triggering this issue. Exception during startup: Unable to gossip with any seeds - Key: CASSANDRA-8072 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 Project: Cassandra Issue Type: Bug Reporter: Ryan Springer Assignee: Brandon Williams Attachments: casandra-system-log-with-assert-patch.log When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*. The error in the /var/log/cassandra/system.log is: ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279) Announcing shutdown INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line 701) Waiting for messaging service to quiesce INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line 941) MessagingService has terminated the accept() thread This errors does not always occur when provisioning a 2-node cluster, but probably around half of the time on only one of the nodes. I haven't been able to reproduce this error with DSC 2.0.9, and there have been no code or definition file changes in Opscenter. I can reproduce locally with the above steps.  I'm happy to test any proposed fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
[ https://issues.apache.org/jira/browse/CASSANDRA-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8684: -- Attachment: Sample.java !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1480884345format=image! !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1206341035format=image! The ever inscrutable results for OS X. I don't buy for a second that it is doing 13 gigabytes/second. !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1911364989format=image! I think the real speed is 3 gigabytes/second which is what I have seen in the past and seen on Linux. There are faster hashes like xxhash or MurmurHash3 to consider that operate in the 5-6 gigabyte/second range. However a Java implementation of xxhash might not hit those numbers. The murmur3 implementation certainly doesn't. A native implementation incurs JNI overhead and there is nothing packaged at the moment. Replace usage of Adler32 with CRC32 --- Key: CASSANDRA-8684 URL: https://issues.apache.org/jira/browse/CASSANDRA-8684 Project: Cassandra Issue Type: Bug Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Attachments: CRCBenchmark.java, PureJavaCrc32.java, Sample.java I could not find a situation in which Adler32 outperformed PureJavaCrc32 much less the intrinsic from Java 8. For small allocations PureJavaCrc32 was much faster probably due to the JNI overhead of invoking the native Adler32 implementation where the array has to be allocated and copied. I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as well as a c3.8xlarge running Ubuntu 14.04. I think it makes sense to stop using Adler32 when generating new checksums. c3.8xlarge, results are time in milliseconds, lower is better ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|47636|46075|25782| |128|36755|36712|23782| |256|31194|32211|22731| |1024|27194|28792|22010| |1048576|25941|27807|21808| |536870912|25957|27840|21836| i5 ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|50539|50466|26826| |128|37092|38533|24553| |256|30630|32938|23459| |1024|26064|29079|22592| |1048576|24357|27911|22481| |536870912|24838|28360|22853| Another fun fact. Performance of the CRC32 intrinsic appears to double from Sandy Bridge - Haswell. Unless I am measuring something different when going from Linux/Sandy to Haswell/OS X. The intrinsic/JDK 8 implementation also operates against DirectByteBuffers better and coding against the wrapper will get that boost when run with Java 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[ https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292329#comment-14292329 ] Mark Cooper commented on CASSANDRA-8061: Ok, I am not running with this branch on one machine and am seeing this non-fatal assert in the logs pretty continuously. I'm not sure if this is related: INFO [CompactionExecutor:157] 2015-01-26 19:57:26,846 ColumnFamilyStore.java:857 - Enqueuing flush of compactions_in_progress: 165 (0%) on-heap, 0 (0%) off-heap INFO [MemtableFlushWriter:45] 2015-01-26 19:57:26,847 Memtable.java:339 - Writing Memtable-compactions_in_progress@1222533013(0 serialized bytes, 1 ops, 0%/0% of on/off-heap limit) INFO [MemtableFlushWriter:45] 2015-01-26 19:57:26,852 Memtable.java:378 - Completed flushing /mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-43-Data.db (42 bytes) for commitlog position ReplayPosition(segmentId=142230151, position=26331623) INFO [CompactionExecutor:131] 2015-01-26 19:57:26,853 CompactionTask.java:136 - Compacting [SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-40-Data.db'), SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-42-Data.db'), SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-41-Data.db'), SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-43-Data.db')] ERROR [CompactionExecutor:157] 2015-01-26 19:57:26,854 CassandraDaemon.java:170 - Exception in thread Thread[CompactionExecutor:157,1,main] java.lang.AssertionError: null at org.apache.cassandra.io.util.Memory.size(Memory.java:307) ~[main/:na] at org.apache.cassandra.utils.obs.OffHeapBitSet.capacity(OffHeapBitSet.java:61) ~[main/:na] at org.apache.cassandra.utils.BloomFilter.indexes(BloomFilter.java:74) ~[main/:na] at org.apache.cassandra.utils.BloomFilter.add(BloomFilter.java:89) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:618) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:187) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:210) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:111) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:188) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:228) ~[main/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_25] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) ~[na:1.7.0_25] at java.util.concurrent.FutureTask.run(FutureTask.java:166) ~[na:1.7.0_25] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_25] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_25] at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] Suppressed: java.lang.AssertionError: null at org.apache.cassandra.io.util.Memory.size(Memory.java:307) ~[main/:na] at org.apache.cassandra.utils.obs.OffHeapBitSet.serialize(OffHeapBitSet.java:108) ~[main/:na] at org.apache.cassandra.utils.BloomFilterSerializer.serialize(BloomFilterSerializer.java:35) ~[main/:na] at org.apache.cassandra.utils.Murmur3BloomFilter$Murmur3BloomFilterSerializer.serialize(Murmur3BloomFilter.java:50) ~[main/:na] at org.apache.cassandra.utils.FilterFactory.serialize(FilterFactory.java:40) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.close(SSTableWriter.java:657) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:512) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:466) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.switchWriter(SSTableRewriter.java:288) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.abort(SSTableRewriter.java:179) ~[main/:na] at
[jira] [Updated] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
[ https://issues.apache.org/jira/browse/CASSANDRA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-8687: --- Attachment: CASSANDRA-8687-trunk.txt Patch for trunk Keyspace should also check Config.isClientMode -- Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Priority: Minor Fix For: 2.1.3, 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt, CASSANDRA-8687-trunk.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
Jeremiah Jordan created CASSANDRA-8687: -- Summary: Keyspace should also check Config.isClientMode Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Priority: Minor Fix For: 2.1.3, 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
[ https://issues.apache.org/jira/browse/CASSANDRA-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292316#comment-14292316 ] Ariel Weisberg edited comment on CASSANDRA-8684 at 1/26/15 8:02 PM: !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1480884345format=image! !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1206341035format=image! The ever inscrutable results for OS X. I am very skeptical it is doing 13 gigabytes/second on a single core. I don't get why there is a gradual increase in speed as the size of the data being checksummed increases, and that speed up doesn't exist on Linux. Looking at a CPU monitor doesn't make it look like the application is using multiple cores. !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1911364989format=image! I think the real speed is 3 gigabytes/second which is what I have seen in the past and seen on Linux. There are faster hashes like xxhash or MurmurHash3 to consider that operate in the 5-6 gigabyte/second range. However a Java implementation of xxhash might not hit those numbers. The murmur3 implementation certainly doesn't. A native implementation incurs JNI overhead and there is nothing packaged at the moment. was (Author: aweisberg): !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1480884345format=image! !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1206341035format=image! The ever inscrutable results for OS X. I don't buy for a second that it is doing 13 gigabytes/second. !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1911364989format=image! I think the real speed is 3 gigabytes/second which is what I have seen in the past and seen on Linux. There are faster hashes like xxhash or MurmurHash3 to consider that operate in the 5-6 gigabyte/second range. However a Java implementation of xxhash might not hit those numbers. The murmur3 implementation certainly doesn't. A native implementation incurs JNI overhead and there is nothing packaged at the moment. Replace usage of Adler32 with CRC32 --- Key: CASSANDRA-8684 URL: https://issues.apache.org/jira/browse/CASSANDRA-8684 Project: Cassandra Issue Type: Bug Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Attachments: CRCBenchmark.java, PureJavaCrc32.java, Sample.java I could not find a situation in which Adler32 outperformed PureJavaCrc32 much less the intrinsic from Java 8. For small allocations PureJavaCrc32 was much faster probably due to the JNI overhead of invoking the native Adler32 implementation where the array has to be allocated and copied. I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as well as a c3.8xlarge running Ubuntu 14.04. I think it makes sense to stop using Adler32 when generating new checksums. c3.8xlarge, results are time in milliseconds, lower is better ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|47636|46075|25782| |128|36755|36712|23782| |256|31194|32211|22731| |1024|27194|28792|22010| |1048576|25941|27807|21808| |536870912|25957|27840|21836| i5 ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|50539|50466|26826| |128|37092|38533|24553| |256|30630|32938|23459| |1024|26064|29079|22592| |1048576|24357|27911|22481| |536870912|24838|28360|22853| Another fun fact. Performance of the CRC32 intrinsic appears to double from Sandy Bridge - Haswell. Unless I am measuring something different when going from Linux/Sandy to Haswell/OS X. The intrinsic/JDK 8 implementation also operates against DirectByteBuffers better and coding against the wrapper will get that boost when run with Java 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
[ https://issues.apache.org/jira/browse/CASSANDRA-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8684: -- Issue Type: Improvement (was: Bug) Replace usage of Adler32 with CRC32 --- Key: CASSANDRA-8684 URL: https://issues.apache.org/jira/browse/CASSANDRA-8684 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Attachments: CRCBenchmark.java, PureJavaCrc32.java, Sample.java I could not find a situation in which Adler32 outperformed PureJavaCrc32 much less the intrinsic from Java 8. For small allocations PureJavaCrc32 was much faster probably due to the JNI overhead of invoking the native Adler32 implementation where the array has to be allocated and copied. I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as well as a c3.8xlarge running Ubuntu 14.04. I think it makes sense to stop using Adler32 when generating new checksums. c3.8xlarge, results are time in milliseconds, lower is better ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|47636|46075|25782| |128|36755|36712|23782| |256|31194|32211|22731| |1024|27194|28792|22010| |1048576|25941|27807|21808| |536870912|25957|27840|21836| i5 ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|50539|50466|26826| |128|37092|38533|24553| |256|30630|32938|23459| |1024|26064|29079|22592| |1048576|24357|27911|22481| |536870912|24838|28360|22853| Another fun fact. Performance of the CRC32 intrinsic appears to double from Sandy Bridge - Haswell. Unless I am measuring something different when going from Linux/Sandy to Haswell/OS X. The intrinsic/JDK 8 implementation also operates against DirectByteBuffers better and coding against the wrapper will get that boost when run with Java 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
[ https://issues.apache.org/jira/browse/CASSANDRA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-8687: --- Attachment: (was: CASSANDRA-8687-trunk.txt) Keyspace should also check Config.isClientMode -- Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Priority: Minor Fix For: 2.1.3, 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8677) rpc_interface and listen_interface generate NPE on startup when specified interface doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-8677:
--
Attachment: (was: 8677.patch)
rpc_interface and listen_interface generate NPE on startup when specified
interface doesn't exist
-
Key: CASSANDRA-8677
URL: https://issues.apache.org/jira/browse/CASSANDRA-8677
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
This is just a buggy UI bit.
Initially the error I got was this which is redundant and not well formatted.
{noformat}
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
Exception encountered during startup: null
Fatal configuration error; unable to start. See log for stacktrace.
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
This has no description of the error that occurred. After logging the
exception.
{noformat}
java.lang.NullPointerException: null
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:347)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:102)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
Exceptions thrown in the DatabaseDescriptor should log in a useful way.
This particular error should generate a message without a stack trace since
it is easily recognized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-6985) ReadExecutors should not rely on static StorageProxy
[ https://issues.apache.org/jira/browse/CASSANDRA-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-6985. -- Resolution: Duplicate ReadExecutors should not rely on static StorageProxy Key: CASSANDRA-6985 URL: https://issues.apache.org/jira/browse/CASSANDRA-6985 Project: Cassandra Issue Type: Sub-task Reporter: Edward Capriolo Assignee: Edward Capriolo Priority: Minor Fix For: 3.0 Attachments: CASSANDRA_6985.1.patch All the Read Executor child classes require use of the Storage Proxy to carry out read. We can pass the StorageProxy along in the constructor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8635) STCS cold sstable omission does not handle overwrites without reads
[ https://issues.apache.org/jira/browse/CASSANDRA-8635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-8635: -- Reviewer: Carl Yeksigian (was: Tyler Hobbs) STCS cold sstable omission does not handle overwrites without reads --- Key: CASSANDRA-8635 URL: https://issues.apache.org/jira/browse/CASSANDRA-8635 Project: Cassandra Issue Type: Bug Components: Core Reporter: Tyler Hobbs Assignee: Marcus Eriksson Priority: Critical Fix For: 2.1.3 Attachments: 0001-Include-cold-sstables-in-compactions-if-they-overlap.patch, 8635-v2.patch In 2.1, STCS may omit cold SSTables from compaction (CASSANDRA-6109). If data is regularly overwritten or deleted (but not enough to trigger a single-sstable tombstone purging compaction), data size on disk may continuously grow if: * The table receives very few reads * The reads only touch the newest SSTables Basically, if the overwritten data is never read and there aren't many tombstones, STCS has no incentive to compact the sstables. We should take sstable overlap into consideration as well as coldness to address this case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7447) New sstable format with support for columnar layout
[ https://issues.apache.org/jira/browse/CASSANDRA-7447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-7447: - Fix Version/s: (was: 3.0) 3.1 New sstable format with support for columnar layout --- Key: CASSANDRA-7447 URL: https://issues.apache.org/jira/browse/CASSANDRA-7447 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Labels: performance, storage Fix For: 3.1 Attachments: ngcc-storage.odp, storage_format.pdf h2. Storage Format Proposal C* has come a long way over the past few years, and unfortunately our storage format hasn't kept pace with the data models we are now encouraging people to utilise. This ticket proposes a collections of storage primitives that can be combined to serve these data models more optimally. It would probably help to first state the data model at the most abstract level. We have a fixed three-tier structure: We have the partition key, the clustering columns, and the data columns. Each have their own characteristics and so require their own specialised treatment. I should note that these changes will necessarily be delivered in stages, and that we will be making some assumptions about what the most useful features to support initially will be. Any features not supported will require sticking with the old format until we extend support to all C* functionality. h3. Partition Key * This really has two components: the partition, and the value. Although the partition is primarily used to distribute across nodes, it can also be used to optimise lookups for a given key within a node * Generally partitioning is by hash, and for the moment I want to focus this ticket on the assumption that this is the case * Given this, it makes sense to optimise our storage format to permit O(1) searching of a given partition. It may be possible to achieve this with little overhead based on the fact we store the hashes in order and know they are approximately randomly distributed, as this effectively forms an immutable contiguous split-ordered list (see Shalev/Shavit, or CASSANDRA-7282), so we only need to store an amount of data based on how imperfectly distributed the hashes are, or at worst a single value per block. * This should completely obviate the need for a separate key-cache, which will be relegated to supporting the old storage format only h3. Primary Key / Clustering Columns * Given we have a hierarchical data model, I propose the use of a cache-oblivious trie * The main advantage of the trie is that it is extremely compact and _supports optimally efficient merges with other tries_ so that we can support more efficient reads when multiple sstables are touched * The trie will be preceded by a small amount of related data; the full partition key, a timestamp epoch (for offset-encoding timestamps) and any other partition level optimisation data, such as (potentially) a min/max timestamp to abort merges earlier * Initially I propose to limit the trie to byte-order comparable data types only (the number of which we can expand through translations of the important types that are not currently) * Crucially the trie will also encapsulate any range tombstones, so that these are merged early in the process and avoids re-iterating the same data * Results in true bidirectional streaming without having to read entire range into memory h3. Values There are generally two approaches to storing rows of data: columnar, or row-oriented. The above two data structures can be combined with a value storage scheme that is based on either. However, given the current model we have of reading large 64Kb blocks for any read, I am inclined to focus on columnar support first, as this delivers order-of-magnitude benefits to those users with the correct workload, while for most workloads our 64Kb blocks are large enough to store row-oriented data in a column-oriented fashion without any performance degradation (I'm happy to consign very large row support to phase 2). Since we will most likely target both behaviours eventually, I am currently inclined to suggest that static columns, sets and maps be targeted for a row-oriented release, as they don't naturally fit in a columnar layout without secondary heap-blocks. This may be easier than delivering heap-blocks also, as it keeps both implementations relatively clean. This is certainly open to debate, and I have no doubt there will be conflicting opinions here. Focusing on our columnar layout, the goals are: * Support sparse and dense column storage * Efficient compression of tombstones, timestamps, ttls, etc.
[jira] [Resolved] (CASSANDRA-6063) Rename internal classes and interfaces to represent the modern Cassandra terminology
[ https://issues.apache.org/jira/browse/CASSANDRA-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-6063. -- Resolution: Duplicate Rename internal classes and interfaces to represent the modern Cassandra terminology Key: CASSANDRA-6063 URL: https://issues.apache.org/jira/browse/CASSANDRA-6063 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Priority: Minor Fix For: 3.0 Some of the class names we are using aren't exactly clear to the newcomers. We've already gone through the Table-Keyspace renaming, but it's not enough. Other things to consider: - ColumnFamilyStore - ColumnFamily (non-critical) - -Column - Cell (where appropriate)- done in e50d6af12fabac98e3cb4bd589e2b92212b041f4 - ColumnFamily - Cells - Row - Partition (PartitionFragment?) (where appropriate) - -RowMutation - Mutation- done in 6bbb13b9b0fe62de2a2140055af2ea6968c73ccc -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8574) Gracefully degrade SELECT when there are lots of tombstones
[ https://issues.apache.org/jira/browse/CASSANDRA-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292063#comment-14292063 ] Sylvain Lebresne commented on CASSANDRA-8574: - Which is turns means exposing tombstones (and implementation detail at the end of the day) in the user API. Which I'm really not a fan of. If the goal is to avoid having pages fail due to tombstones, then I think the proper way to handle it would be to limit _internal_ pages based on tombstones, but doing so transparently for the user (in other words, we could potentially end up doing multiple internal query for a single page returned to the user). Doing so is, however, easier said than done. Gracefully degrade SELECT when there are lots of tombstones --- Key: CASSANDRA-8574 URL: https://issues.apache.org/jira/browse/CASSANDRA-8574 Project: Cassandra Issue Type: Improvement Reporter: Jens Rantil Fix For: 3.0 *Background:* There's lots of tooling out there to do BigData analysis on Cassandra clusters. Examples are Spark and Hadoop, which is offered by DSE. The problem with both of these so far, is that a single partition key with too many tombstones can make the query job fail hard. The described scenario happens despite the user setting a rather small FetchSize. I assume this is a common scenario if you have larger rows. *Proposal:* To allow a CQL SELECT to gracefully degrade to only return a smaller batch of results if there are too many tombstones. The tombstones are ordered according to clustering key and one should be able to page through them. Potentially: SELECT * FROM mytable LIMIT 1000 TOMBSTONES; would page through maximum 1000 tombstones, _or_ 1000 (CQL) rows. I understand that this obviously would degrade performance, but it would at least yield a result. *Additional comment:* I haven't dug into Cassandra code, but conceptually I guess this would be doable. Let me know what you think. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6986) snapshot fails with SSTable not found if table has already been restored
[ https://issues.apache.org/jira/browse/CASSANDRA-6986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292092#comment-14292092 ] Alan Boudreault commented on CASSANDRA-6986: [~slowenthal] Have you experienced this issue again? If so, any chance we can talk together to check how to reproduce it? snapshot fails with SSTable not found if table has already been restored Key: CASSANDRA-6986 URL: https://issues.apache.org/jira/browse/CASSANDRA-6986 Project: Cassandra Issue Type: Bug Components: Tools Environment: C* 2.0.6 Reporter: Steven Lowenthal Assignee: Ryan McGuire Priority: Minor Attachments: 6986.tar.gz, test.sh I seem to be running into a weird restore issue. I load a database with sstableloader, and then take a snapshot of the keyspace. I then truncate the tables in the keyspace, replace the sstables from the snapshot, and refresh everything. It seems to work fine. Of course the sstables get renumbered. I then try to create a new backup of the keyspace, and this happens on one of the tables on one of the nodes. (and the sstable has been renumbered): Exception in thread main java.lang.RuntimeException: Tried to hard link to file that does not exist /raid0/cassandra/data/stock/trades/stock-trades-jb-18-Data.db -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-7642:
-
Fix Version/s: (was: 3.0)
3.1
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.1
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8640) Paxos requires all nodes for CAS
[
https://issues.apache.org/jira/browse/CASSANDRA-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8640:
---
Labels: qa-resolved (was: )
Paxos requires all nodes for CAS
Key: CASSANDRA-8640
URL: https://issues.apache.org/jira/browse/CASSANDRA-8640
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Anthony Cozzie
Assignee: Anthony Cozzie
Labels: qa-resolved
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Fix-parentheses-bug-in-Paxos.patch,
0001-Fix-quorum-required-nodes-calculation-for-Paxos.patch
In C* 2.1,
{code}
int requiredParticipants = participants + 1 / 2; // See CASSANDRA-833
{code}
Will always return participants because of operator precedence. I am not
sure just adding parentheses will fix the problem, though, as the original
code differentiated between pending and natural endpoints.
{code}
int requiredParticipants = pendingEndpoints.size() + 1 +
naturalEndpoints.size() / 2; // See CASSANDRA-833
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292129#comment-14292129 ] Ariel Weisberg commented on CASSANDRA-8614: --- I think I may be wrong. There is some wonkiness with the build picking up JDK 8 some of the time even though the first JDK in my path is 7. I think you do need the .class file for the fake CRC32 and not just the source. Select optimal CRC32 implementation at runtime -- Key: CASSANDRA-8614 URL: https://issues.apache.org/jira/browse/CASSANDRA-8614 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Labels: performance Attachments: 8614.patch, Sample.java JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec per core in my quick and dirty test. PureJavaCRC32 is 800 megabytes/sec if I recall and it has a lookup table that evicts random cache lines every time it runs. In order to capture the benefit of that when it is available we can select a CRC32 implementation at startup in a static block. If JDK 8 is not what is running we can fall back to the existing PureJavaCRC32 implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8657) long-test LongCompactionsTest fails
[
https://issues.apache.org/jira/browse/CASSANDRA-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292190#comment-14292190
]
Michael Shuler commented on CASSANDRA-8657:
---
I've tried numerous release tags (2.1.0, 2.0.9, 2.0.3, and a few git log shas),
as well as the first commit in git log for this test, f92fb22, and they all
fail. The first commit fails with a different error:
{noformat}
[junit] Testcase:
testStandardColumnCompactions(org.apache.cassandra.db.compaction.LongCompactionsTest):
FAILED
[junit] expected:9 but was:99
[junit] junit.framework.AssertionFailedError: expected:9 but was:99
[junit] at
org.apache.cassandra.db.compaction.CompactionsTest.assertMaxTimestamp(CompactionsTest.java:208)
[junit] at
org.apache.cassandra.db.compaction.LongCompactionsTest.testStandardColumnCompactions(LongCompactionsTest.java:141)
{noformat}
I haven't been able to find a place that this test runs successfully, so can't
bisect it.
long-test LongCompactionsTest fails
---
Key: CASSANDRA-8657
URL: https://issues.apache.org/jira/browse/CASSANDRA-8657
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Michael Shuler
Priority: Minor
Fix For: 2.1.3, 2.0.13
Attachments: system.log
Same error on 3 of the 4 tests in this suite - failure is the same for 2.0
and 2.1 branch:
{noformat}
[junit] Testsuite: org.apache.cassandra.db.compaction.LongCompactionsTest
[junit] Tests run: 4, Failures: 3, Errors: 0, Skipped: 0, Time elapsed:
27.294 sec
[junit]
[junit] Testcase:
testCompactionMany(org.apache.cassandra.db.compaction.LongCompactionsTest):
FAILED
[junit]
/tmp/Keyspace14247587528884809907Standard1/Keyspace1/Keyspace1-Standard1-jb-0-Data.db
is not correctly marked compacting
[junit] junit.framework.AssertionFailedError:
/tmp/Keyspace14247587528884809907Standard1/Keyspace1/Keyspace1-Standard1-jb-0-Data.db
is not correctly marked compacting
[junit] at
org.apache.cassandra.db.compaction.AbstractCompactionTask.init(AbstractCompactionTask.java:49)
[junit] at
org.apache.cassandra.db.compaction.CompactionTask.init(CompactionTask.java:47)
[junit] at
org.apache.cassandra.db.compaction.LongCompactionsTest.testCompaction(LongCompactionsTest.java:102)
[junit] at
org.apache.cassandra.db.compaction.LongCompactionsTest.testCompactionMany(LongCompactionsTest.java:67)
[junit]
[junit]
[junit] Testcase:
testCompactionSlim(org.apache.cassandra.db.compaction.LongCompactionsTest):
FAILED
[junit]
/tmp/Keyspace13809058557206351042Standard1/Keyspace1/Keyspace1-Standard1-jb-0-Data.db
is not correctly marked compacting
[junit] junit.framework.AssertionFailedError:
/tmp/Keyspace13809058557206351042Standard1/Keyspace1/Keyspace1-Standard1-jb-0-Data.db
is not correctly marked compacting
[junit] at
org.apache.cassandra.db.compaction.AbstractCompactionTask.init(AbstractCompactionTask.java:49)
[junit] at
org.apache.cassandra.db.compaction.CompactionTask.init(CompactionTask.java:47)
[junit] at
org.apache.cassandra.db.compaction.LongCompactionsTest.testCompaction(LongCompactionsTest.java:102)
[junit] at
org.apache.cassandra.db.compaction.LongCompactionsTest.testCompactionSlim(LongCompactionsTest.java:58)
[junit]
[junit]
[junit] Testcase:
testCompactionWide(org.apache.cassandra.db.compaction.LongCompactionsTest):
FAILED
[junit]
/tmp/Keyspace15276133158440321595Standard1/Keyspace1/Keyspace1-Standard1-jb-0-Data.db
is not correctly marked compacting
[junit] junit.framework.AssertionFailedError:
/tmp/Keyspace15276133158440321595Standard1/Keyspace1/Keyspace1-Standard1-jb-0-Data.db
is not correctly marked compacting
[junit] at
org.apache.cassandra.db.compaction.AbstractCompactionTask.init(AbstractCompactionTask.java:49)
[junit] at
org.apache.cassandra.db.compaction.CompactionTask.init(CompactionTask.java:47)
[junit] at
org.apache.cassandra.db.compaction.LongCompactionsTest.testCompaction(LongCompactionsTest.java:102)
[junit] at
org.apache.cassandra.db.compaction.LongCompactionsTest.testCompactionWide(LongCompactionsTest.java:49)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.compaction.LongCompactionsTest FAILED
{noformat}
A system.log is attached from the above run on 2.0 HEAD.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-6065) Use CQL3 internally in schema code and HHOM
[ https://issues.apache.org/jira/browse/CASSANDRA-6065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-6065. -- Resolution: Duplicate Use CQL3 internally in schema code and HHOM --- Key: CASSANDRA-6065 URL: https://issues.apache.org/jira/browse/CASSANDRA-6065 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Priority: Minor Fix For: 3.0 We mostly use CQL3 internally everywhere now, except HHOM and schema-related code. We should switch to CQL3+the new paging for HHOM to replace the current ugliness and to CQL3 for all schema-related serialization and deserialization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8677) rpc_interface and listen_interface generate NPE on startup when specified interface doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-8677:
--
Attachment: (was: 8677-2.1.patch)
rpc_interface and listen_interface generate NPE on startup when specified
interface doesn't exist
-
Key: CASSANDRA-8677
URL: https://issues.apache.org/jira/browse/CASSANDRA-8677
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
This is just a buggy UI bit.
Initially the error I got was this which is redundant and not well formatted.
{noformat}
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
Exception encountered during startup: null
Fatal configuration error; unable to start. See log for stacktrace.
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
This has no description of the error that occurred. After logging the
exception.
{noformat}
java.lang.NullPointerException: null
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:347)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:102)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
Exceptions thrown in the DatabaseDescriptor should log in a useful way.
This particular error should generate a message without a stack trace since
it is easily recognized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8677) rpc_interface and listen_interface generate NPE on startup when specified interface doesn't exist
[
https://issues.apache.org/jira/browse/CASSANDRA-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-8677:
--
Attachment: 8677.patch
8677-2.1.patch
Updated patch incorporating Robert's feedback.
rpc_interface and listen_interface generate NPE on startup when specified
interface doesn't exist
-
Key: CASSANDRA-8677
URL: https://issues.apache.org/jira/browse/CASSANDRA-8677
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Attachments: 8677-2.1.patch, 8677.patch
This is just a buggy UI bit.
Initially the error I got was this which is redundant and not well formatted.
{noformat}
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
Exception encountered during startup: null
Fatal configuration error; unable to start. See log for stacktrace.
ERROR 20:12:55 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
Fatal configuration error; unable to start. See log for stacktrace.
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:108)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
This has no description of the error that occurred. After logging the
exception.
{noformat}
java.lang.NullPointerException: null
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:347)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:102)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:122)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:479)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:571)
[main/:na]
{noformat}
Exceptions thrown in the DatabaseDescriptor should log in a useful way.
This particular error should generate a message without a stack trace since
it is easily recognized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292168#comment-14292168
]
Joshua McKenzie commented on CASSANDRA-8535:
[~maximp] Separate tickets, but before you do that - have you run a [nodetool
scrub|http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsScrub.html]
against the offending keyspace? the Index summary build failure is an
assertion that it doesn't have any keys to write. We could (probably should)
move that assertion to earlier when the List is created to better indicate the
failed path than during the build() call - please create a separate ticket for
that stack and try out the scrub and report back on there.
As for your 2nd exception, looks like you have mismatched C* nodes trying to
speak to one another.
{code:title=Frame exception}
if (version Server.CURRENT_VERSION)
throw new ProtocolException(Invalid or unsupported protocol version: +
version);
{code}
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8535_v1.txt
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it is being used by another
process.
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_45]
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
~[na:1.7.0_45]
at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
at
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
~[main/:na]
... 18 common frames omitted
{code}
--
This message was sent by
[jira] [Comment Edited] (CASSANDRA-8061) tmplink files are not removed
[ https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292329#comment-14292329 ] Mark Cooper edited comment on CASSANDRA-8061 at 1/26/15 8:49 PM: - Ok, I am now running with this branch on one machine and am seeing this non-fatal assert in the logs pretty continuously. I'm not sure if this is related: INFO [CompactionExecutor:157] 2015-01-26 19:57:26,846 ColumnFamilyStore.java:857 - Enqueuing flush of compactions_in_progress: 165 (0%) on-heap, 0 (0%) off-heap INFO [MemtableFlushWriter:45] 2015-01-26 19:57:26,847 Memtable.java:339 - Writing Memtable-compactions_in_progress@1222533013(0 serialized bytes, 1 ops, 0%/0% of on/off-heap limit) INFO [MemtableFlushWriter:45] 2015-01-26 19:57:26,852 Memtable.java:378 - Completed flushing /mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-43-Data.db (42 bytes) for commitlog position ReplayPosition(segmentId=142230151, position=26331623) INFO [CompactionExecutor:131] 2015-01-26 19:57:26,853 CompactionTask.java:136 - Compacting [SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-40-Data.db'), SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-42-Data.db'), SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-41-Data.db'), SSTableReader(path='/mnt/flash1/cass0/data/system/compactions_in_progress/system-compactions_in_progress-ka-43-Data.db')] ERROR [CompactionExecutor:157] 2015-01-26 19:57:26,854 CassandraDaemon.java:170 - Exception in thread Thread[CompactionExecutor:157,1,main] java.lang.AssertionError: null at org.apache.cassandra.io.util.Memory.size(Memory.java:307) ~[main/:na] at org.apache.cassandra.utils.obs.OffHeapBitSet.capacity(OffHeapBitSet.java:61) ~[main/:na] at org.apache.cassandra.utils.BloomFilter.indexes(BloomFilter.java:74) ~[main/:na] at org.apache.cassandra.utils.BloomFilter.add(BloomFilter.java:89) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:618) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:187) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:210) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:111) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:188) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:228) ~[main/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_25] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) ~[na:1.7.0_25] at java.util.concurrent.FutureTask.run(FutureTask.java:166) ~[na:1.7.0_25] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_25] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_25] at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] Suppressed: java.lang.AssertionError: null at org.apache.cassandra.io.util.Memory.size(Memory.java:307) ~[main/:na] at org.apache.cassandra.utils.obs.OffHeapBitSet.serialize(OffHeapBitSet.java:108) ~[main/:na] at org.apache.cassandra.utils.BloomFilterSerializer.serialize(BloomFilterSerializer.java:35) ~[main/:na] at org.apache.cassandra.utils.Murmur3BloomFilter$Murmur3BloomFilterSerializer.serialize(Murmur3BloomFilter.java:50) ~[main/:na] at org.apache.cassandra.utils.FilterFactory.serialize(FilterFactory.java:40) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.close(SSTableWriter.java:657) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:512) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:466) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.switchWriter(SSTableRewriter.java:288) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.abort(SSTableRewriter.java:179)
[jira] [Created] (CASSANDRA-8688) Standalone sstableupgrade tool throws exception
Yuki Morishita created CASSANDRA-8688:
-
Summary: Standalone sstableupgrade tool throws exception
Key: CASSANDRA-8688
URL: https://issues.apache.org/jira/browse/CASSANDRA-8688
Project: Cassandra
Issue Type: Bug
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Minor
Fix For: 2.1.3
Attachments:
0001-remove-size-calculation-from-standalone-sstbaleupgra.patch
Standalone sstableupgrade throws following exception:
{code}
Exception in thread main java.lang.AssertionError: Incoherent new size -1
replacing
[SSTableReader(path='/cassandra/bin/./../data/data/ks/cf-0f578640a59211e4a5a2ef9f87394ca6/ks-cf-jb-53881-Data.db')]
by [] in View(pending_count=0, sstables=[], compacting=[])
at org.apache.cassandra.db.DataTracker$View.newSSTables(DataTracker.java:741)
at org.apache.cassandra.db.DataTracker$View.replace(DataTracker.java:713)
at
org.apache.cassandra.db.DataTracker.removeSSTablesFromTracker(DataTracker.java:418)
at org.apache.cassandra.db.DataTracker.markCompactedSSTablesReplaced(
DataTracker.java:273)
at org.apache.cassandra.db.compaction.Upgrader.upgrade(Upgrader.java:96)
{code}
Upgrader is trying to calculate SSTables size dispite the tool is meant to be
used offline.
This is introduced in CASSANDRA-7852 and only affects cassandra 2.1.2.
Online upgrade (nodetool upgradesstables) is not affected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data
[
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292451#comment-14292451
]
Randy Fradin commented on CASSANDRA-6285:
-
I am getting this exception using Thrift HSHA in 2.1.0:
{quote}
INFO [CompactionExecutor:8] 2015-01-26 13:32:51,818 CompactionTask.java (line
138) Compacting
[SSTableReader(path='/tmp/cass_test/cassandra/TestCassandra/data/test_ks/test_cf-1c45da40a58911e4826751fbbc77b187/test_ks-test_cf-ka-2-Data.db'),
SSTableReader(path='/tmp/cass_test/cassandra/TestCassandra/data/test_ks/test_cf-1c45da40a58911e4826751fbbc77b187/test_ks-test_cf-ka-1-Data.db')]
INFO [CompactionExecutor:8] 2015-01-26 13:32:51,890 ColumnFamilyStore.java
(line 856) Enqueuing flush of compactions_in_progress: 212 (0%) on-heap, 20
(0%) off-heap
INFO [MemtableFlushWriter:8] 2015-01-26 13:32:51,892 Memtable.java (line 326)
Writing Memtable-compactions_in_progress@1155018639(0 serialized bytes, 1 ops,
0%/0% of on/off-heap limit)
INFO [MemtableFlushWriter:8] 2015-01-26 13:32:51,896 Memtable.java (line 360)
Completed flushing
/tmp/cass_test/cassandra/TestCassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-2-Data.db
(42 bytes) for commitlog position ReplayPosition(segmentId=1422296630707,
position=430226)
ERROR [CompactionExecutor:8] 2015-01-26 13:32:51,906 CassandraDaemon.java (line
166) Exception in thread Thread[CompactionExecutor:8,1,RMI Runtime]
java.lang.RuntimeException: Last written key
DecoratedKey(131206587314004820534098544948237170809,
80010001000c62617463685f6d757461746500) = current key
DecoratedKey(14775611966645399672119169777260659240,
726f776b65793030385f31343232323937313537353835) writing into
/tmp/cass_test/cassandra/TestCassandra/data/test_ks/test_cf-1c45da40a58911e4826751fbbc77b187/test_ks-test_cf-tmp-ka-3-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:196)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:110)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:177)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:235)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_40]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_40]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_40]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_40]
at java.lang.Thread.run(Thread.java:724) [na:1.7.0_40]
{quote}
I don't think it's caused by CASSANDRA-8211, because it happens during the
first compaction that takes place between the first 2 SSTables to get flushed
from an initially empty column family.
Also, I've only been able to reproduce it when using both *hsha* for the rpc
server and *offheap_objects* for memtable allocation. If I switch either to
sync or to offheap_buffers or heap_buffers then I cannot reproduce the problem.
Also under the same circumstances I'm pretty sure I've seen incorrect data
being returned to a client multiget_slice request before any SSTables had been
flushed yet, so I presume this is corruption that happens before any
flush/compaction takes place.
nodetool scrub yielded these errors:
{quote}
INFO [CompactionExecutor:9] 2015-01-26 13:48:01,512 OutputHandler.java (line
42) Scrubbing
SSTableReader(path='/tmp/cass_test/cassandra/TestCassandra/data/test_ks/test_cf-1c45da40a58911e4826751fbbc77b187/test_ks-test_cf-ka-2-Data.db')
(168780 bytes)
INFO [CompactionExecutor:10] 2015-01-26 13:48:01,512 OutputHandler.java (line
42) Scrubbing
SSTableReader(path='/tmp/cass_test/cassandra/TestCassandra/data/test_ks/test_cf-1c45da40a58911e4826751fbbc77b187/test_ks-test_cf-ka-1-Data.db')
(135024 bytes)
WARN [CompactionExecutor:9] 2015-01-26 13:48:01,531 OutputHandler.java (line
52) Out of
[jira] [Created] (CASSANDRA-8689) Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
Jeff Liu created CASSANDRA-8689: --- Summary: Assertion error in 2.1.2: ERROR [IndexSummaryManager:1] Key: CASSANDRA-8689 URL: https://issues.apache.org/jira/browse/CASSANDRA-8689 Project: Cassandra Issue Type: Bug Reporter: Jeff Liu After upgrading a 6 nodes cassandra from 2.1.0 to 2.1.2, start getting the following assertion error. /noformat ERROR [IndexSummaryManager:1] 2015-01-26 20:55:40,451 CassandraDaemon.java:153 - Exception in thread Thread[IndexSummaryManager:1,1,main] java.lang.AssertionError: null at org.apache.cassandra.io.util.Memory.size(Memory.java:307) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.io.sstable.IndexSummary.getOffHeapSize(IndexSummary.java:192) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.io.sstable.SSTableReader.getIndexSummaryOffHeapSize(SSTableReader.java:1070) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:292) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:77) ~[apache-cassandra-2.1.2.jar:2.1.2] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_45] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_45] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_45] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] /noformat -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292404#comment-14292404
]
Mark Cooper edited comment on CASSANDRA-8061 at 1/26/15 9:01 PM:
-
[~JoshuaMcKenzie] - oops yes, meant to say I am now running this patch.
After reading through the patch I added this to bin/cassandra:
+cassandra_parms=$cassandra_parms -Dcassandra.debugrefcount=true
I assume I'll see things like:
BAD RELEASE...
LEAK DETECTED...
...in the logs. So far I don't see this.
was (Author: markncooper):
[~JoshuaMcKenzie] - oops yes, meant to say I am now running this patch.
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
[jira] [Updated] (CASSANDRA-8689) Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
[
https://issues.apache.org/jira/browse/CASSANDRA-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Liu updated CASSANDRA-8689:
Description:
After upgrading a 6 nodes cassandra from 2.1.0 to 2.1.2, start getting the
following assertion error.
{noformat}
ERROR [IndexSummaryManager:1] 2015-01-26 20:55:40,451 CassandraDaemon.java:153
- Exception in thread Thread[IndexSummaryManager:1,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummary.getOffHeapSize(IndexSummary.java:192)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.SSTableReader.getIndexSummaryOffHeapSize(SSTableReader.java:1070)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:292)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:77)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{noformat}
cassandra service is still running despite the issue. Node has total 8G memory
with 2G allocated to heap. We are basically running read queries to retrieve
data out of cassandra.
was:
After upgrading a 6 nodes cassandra from 2.1.0 to 2.1.2, start getting the
following assertion error.
{noformat}
ERROR [IndexSummaryManager:1] 2015-01-26 20:55:40,451 CassandraDaemon.java:153
- Exception in thread Thread[IndexSummaryManager:1,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummary.getOffHeapSize(IndexSummary.java:192)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.SSTableReader.getIndexSummaryOffHeapSize(SSTableReader.java:1070)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:292)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:77)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{noformat}
cassandra service is still running despite the issue. Node has total 8G memory
with 2G allocated to heap.
Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
[jira] [Updated] (CASSANDRA-8689) Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
[
https://issues.apache.org/jira/browse/CASSANDRA-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Liu updated CASSANDRA-8689:
Description:
After upgrading a 6 nodes cassandra from 2.1.0 to 2.1.2, start getting the
following assertion error.
{noformat}
ERROR [IndexSummaryManager:1] 2015-01-26 20:55:40,451 CassandraDaemon.java:153
- Exception in thread Thread[IndexSummaryManager:1,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummary.getOffHeapSize(IndexSummary.java:192)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.SSTableReader.getIndexSummaryOffHeapSize(SSTableReader.java:1070)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:292)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:77)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{noformat}
cassandra service is still running despite the issue. Node has total 8G memory
with 2G allocated to heap.
was:
After upgrading a 6 nodes cassandra from 2.1.0 to 2.1.2, start getting the
following assertion error.
/noformat
ERROR [IndexSummaryManager:1] 2015-01-26 20:55:40,451 CassandraDaemon.java:153
- Exception in thread Thread[IndexSummaryManager:1,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummary.getOffHeapSize(IndexSummary.java:192)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.SSTableReader.getIndexSummaryOffHeapSize(SSTableReader.java:1070)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:292)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:77)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
/noformat
Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
---
Key: CASSANDRA-8689
URL: https://issues.apache.org/jira/browse/CASSANDRA-8689
Project: Cassandra
Issue
[jira] [Updated] (CASSANDRA-8689) Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
[
https://issues.apache.org/jira/browse/CASSANDRA-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8689:
---
Reproduced In: 2.1.2
Fix Version/s: 2.1.3
Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
---
Key: CASSANDRA-8689
URL: https://issues.apache.org/jira/browse/CASSANDRA-8689
Project: Cassandra
Issue Type: Bug
Reporter: Jeff Liu
Fix For: 2.1.3
After upgrading a 6 nodes cassandra from 2.1.0 to 2.1.2, start getting the
following assertion error.
{noformat}
ERROR [IndexSummaryManager:1] 2015-01-26 20:55:40,451
CassandraDaemon.java:153 - Exception in thread
Thread[IndexSummaryManager:1,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummary.getOffHeapSize(IndexSummary.java:192)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.SSTableReader.getIndexSummaryOffHeapSize(SSTableReader.java:1070)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:292)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:77)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.7.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
[na:1.7.0_45]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{noformat}
cassandra service is still running despite the issue. Node has total 8G
memory with 2G allocated to heap. We are basically running read queries to
retrieve data out of cassandra.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8666) Simplify logic of ABSC#BatchRemoveIterator#commit()
[ https://issues.apache.org/jira/browse/CASSANDRA-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292377#comment-14292377 ] Joshua McKenzie commented on CASSANDRA-8666: +1 Simplify logic of ABSC#BatchRemoveIterator#commit() --- Key: CASSANDRA-8666 URL: https://issues.apache.org/jira/browse/CASSANDRA-8666 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Priority: Trivial Fix For: 2.1.3, 2.0.13 Attachments: 8666.txt Replace current logic with a straightforward single loop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292396#comment-14292396
]
Joshua McKenzie commented on CASSANDRA-8061:
[~markncooper] Did you mean you are now running this patch?
I've seen the null assertion on Memory.size when making changes to ordering of
SSTableReader cleanup / deletion / SSTableRewriter stuff locally so it'd make a
bit more sense if you're seeing it with the other branch rather than on 2.1.2.
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
113M
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292400#comment-14292400
]
Joshua McKenzie commented on CASSANDRA-8061:
/cc [~benedict]: how confident are you as to the current state of your 7705-2.1
branch?
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
113M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Data.db
116M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Data.db
cassandra git commit: trunk is using 1.0.5.2 of snappy-java
Repository: cassandra
Updated Branches:
refs/heads/trunk 0b08c994c - 7fb99d93f
trunk is using 1.0.5.2 of snappy-java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7fb99d93
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7fb99d93
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7fb99d93
Branch: refs/heads/trunk
Commit: 7fb99d93ff32a9e972d9f359705e123e6b72e0f6
Parents: 0b08c99
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Jan 27 02:05:04 2015 -0500
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Jan 27 02:05:04 2015 -0500
--
build.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7fb99d93/build.xml
--
diff --git a/build.xml b/build.xml
index 9bd565f..12c877a 100644
--- a/build.xml
+++ b/build.xml
@@ -316,7 +316,7 @@
license name=The Apache Software License, Version 2.0
url=http://www.apache.org/licenses/LICENSE-2.0.txt/
scm connection=${scm.connection}
developerConnection=${scm.developerConnection} url=${scm.url}/
dependencyManagement
- dependency groupId=org.xerial.snappy artifactId=snappy-java
version=1.0.5/
+ dependency groupId=org.xerial.snappy artifactId=snappy-java
version=1.0.5.2/
dependency groupId=net.jpountz.lz4 artifactId=lz4
version=1.3.0/
dependency groupId=com.ning artifactId=compress-lzf
version=0.8.4/
dependency groupId=com.google.guava artifactId=guava
version=16.0/
[jira] [Assigned] (CASSANDRA-8685) Consider upgrade to thrift 0.9.2 (or later)
[ https://issues.apache.org/jira/browse/CASSANDRA-8685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-8685: - Assignee: T Jake Luciani Consider upgrade to thrift 0.9.2 (or later) --- Key: CASSANDRA-8685 URL: https://issues.apache.org/jira/browse/CASSANDRA-8685 Project: Cassandra Issue Type: Bug Reporter: Adam Hattrell Assignee: T Jake Luciani Folks using Astyanax and the like are subject to https://issues.apache.org/jira/browse/THRIFT-1457 and may run into heap pressure on the Cassandra side for larger read request, as thrift doesn't reset its internal buffer. This can lead to larger TFramedTransport instances will be kept on the heap. I've seen at least one situation where this has saved around 1Gb of heap space on average. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7933) Update cassandra-stress README
[ https://issues.apache.org/jira/browse/CASSANDRA-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292673#comment-14292673 ] Philip Thompson commented on CASSANDRA-7933: So given that we are delaying a full guide, is this good enough to put into 2.1.3? The vote will probably be happening this week. Update cassandra-stress README -- Key: CASSANDRA-7933 URL: https://issues.apache.org/jira/browse/CASSANDRA-7933 Project: Cassandra Issue Type: Task Reporter: Benedict Assignee: Philip Thompson Priority: Minor Fix For: 2.1.3 Attachments: 7933-2.txt, CASSANDRA-7933.txt There is a README in the tools/stress directory. It is completely out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8690) Ensure all error handling code that may itself throw an exception utilises addSuppressed
Benedict created CASSANDRA-8690: --- Summary: Ensure all error handling code that may itself throw an exception utilises addSuppressed Key: CASSANDRA-8690 URL: https://issues.apache.org/jira/browse/CASSANDRA-8690 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Priority: Minor Fix For: 2.1.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: using commons-lang3, not commons-lang
Repository: cassandra Updated Branches: refs/heads/trunk 7fb99d93f - 6068daea8 using commons-lang3, not commons-lang Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6068daea Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6068daea Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6068daea Branch: refs/heads/trunk Commit: 6068daea866bffd9fcc20f81219a21631dbf9bf0 Parents: 7fb99d9 Author: Dave Brosius dbros...@mebigfatguy.com Authored: Tue Jan 27 02:28:04 2015 -0500 Committer: Dave Brosius dbros...@mebigfatguy.com Committed: Tue Jan 27 02:28:04 2015 -0500 -- test/unit/org/apache/cassandra/cql3/CreateIndexStatementTest.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6068daea/test/unit/org/apache/cassandra/cql3/CreateIndexStatementTest.java -- diff --git a/test/unit/org/apache/cassandra/cql3/CreateIndexStatementTest.java b/test/unit/org/apache/cassandra/cql3/CreateIndexStatementTest.java index 847466e..1e2e084 100644 --- a/test/unit/org/apache/cassandra/cql3/CreateIndexStatementTest.java +++ b/test/unit/org/apache/cassandra/cql3/CreateIndexStatementTest.java @@ -19,7 +19,7 @@ package org.apache.cassandra.cql3; import java.util.Locale; -import org.apache.commons.lang.StringUtils; +import org.apache.commons.lang3.StringUtils; import org.junit.Test;
[jira] [Commented] (CASSANDRA-7688) Add data sizing to a system table
[
https://issues.apache.org/jira/browse/CASSANDRA-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291621#comment-14291621
]
mck commented on CASSANDRA-7688:
{quote}It would be an on-demand calculation that would be moderately expensive.
{quote}
[~iamaleksey] If the implementation is but a rewrite, i'm also keen on [~mbyrd]
question.
{quote} Can you please elaborate on what the idea is behind storing this info
in a system table?{quote}
Add data sizing to a system table
-
Key: CASSANDRA-7688
URL: https://issues.apache.org/jira/browse/CASSANDRA-7688
Project: Cassandra
Issue Type: New Feature
Reporter: Jeremiah Jordan
Assignee: Aleksey Yeschenko
Fix For: 2.1.3
Currently you can't implement something similar to describe_splits_ex purely
from the a native protocol driver.
https://datastax-oss.atlassian.net/browse/JAVA-312 is open to expose easily
getting ownership information to a client in the java-driver. But you still
need the data sizing part to get splits of a given size. We should add the
sizing information to a system table so that native clients can get to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-7688) Add data sizing to a system table
[
https://issues.apache.org/jira/browse/CASSANDRA-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-7688:
---
Comment: was deleted
(was: {quote}It would be an on-demand calculation that would be moderately
expensive. {quote}
[~iamaleksey] If the implementation is but a rewrite, i'm also keen on [~mbyrd]
question.
{quote} Can you please elaborate on what the idea is behind storing this info
in a system table?{quote})
Add data sizing to a system table
-
Key: CASSANDRA-7688
URL: https://issues.apache.org/jira/browse/CASSANDRA-7688
Project: Cassandra
Issue Type: New Feature
Reporter: Jeremiah Jordan
Assignee: Aleksey Yeschenko
Fix For: 2.1.3
Currently you can't implement something similar to describe_splits_ex purely
from the a native protocol driver.
https://datastax-oss.atlassian.net/browse/JAVA-312 is open to expose easily
getting ownership information to a client in the java-driver. But you still
need the data sizing part to get splits of a given size. We should add the
sizing information to a system table so that native clients can get to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7688) Add data sizing to a system table
[
https://issues.apache.org/jira/browse/CASSANDRA-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291619#comment-14291619
]
mck commented on CASSANDRA-7688:
{quote}It would be an on-demand calculation that would be moderately expensive.
{quote}
[~iamaleksey] If the implementation is but a rewrite, i'm also keen on [~mbyrd]
question.
{quote} Can you please elaborate on what the idea is behind storing this info
in a system table?{quote}
Add data sizing to a system table
-
Key: CASSANDRA-7688
URL: https://issues.apache.org/jira/browse/CASSANDRA-7688
Project: Cassandra
Issue Type: New Feature
Reporter: Jeremiah Jordan
Assignee: Aleksey Yeschenko
Fix For: 2.1.3
Currently you can't implement something similar to describe_splits_ex purely
from the a native protocol driver.
https://datastax-oss.atlassian.net/browse/JAVA-312 is open to expose easily
getting ownership information to a client in the java-driver. But you still
need the data sizing part to get splits of a given size. We should add the
sizing information to a system table so that native clients can get to it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8623) sstablesplit fails *randomly* with Data component is missing
[
https://issues.apache.org/jira/browse/CASSANDRA-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291547#comment-14291547
]
Marcus Eriksson edited comment on CASSANDRA-8623 at 1/26/15 8:17 AM:
-
[~aboudreault] could you test this patch? I have not been able to reproduce
this myself
Patch just makes sure all compactions are finished before waiting for the
deletion tasks
Same issue could be in offline cleanup and offline scrub
oh btw, you can make the offline tools log more by changing WARN to DEBUG in
conf/logback-tools.xml if it still fails
was (Author: krummas):
[~aboudreault] could you test this patch? I have not been able to reproduce
this myself
Patch just makes sure all compactions are finished before waiting for the
deletion tasks
Same issue could be in offline cleanup and offline scrub
sstablesplit fails *randomly* with Data component is missing
Key: CASSANDRA-8623
URL: https://issues.apache.org/jira/browse/CASSANDRA-8623
Project: Cassandra
Issue Type: Bug
Reporter: Alan Boudreault
Assignee: Marcus Eriksson
Attachments:
0001-make-sure-we-finish-compactions-before-waiting-for-d.patch, output.log
I'm experiencing an issue related to sstablesplit. I would like to understand
if I am doing something wrong or there is an issue in the split process. The
process fails randomly with the following exception:
{code}
ERROR 02:17:36 Error in ThreadPoolExecutor
java.lang.AssertionError: Data component is missing for
sstable./tools/bin/../../data/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-16
{code}
See attached output.log file. The process never stops after this exception
and I've also seen the dataset growing indefinitely (number of sstables).
* I have not been able to reproduce the issue with a single sstablesplit
command. ie, specifying all files with glob matching.
* I can reproduce the bug if I call multiple sstablesplit one file at the
time (the way ccm does)
Here is the test case file to reproduce the bug:
https://drive.google.com/file/d/0BwZ_GPM33j6KdVh0NTdkOWV2R1E/view?usp=sharing
1. Download the split_issue.tar.gz file. It includes latest cassandra-2.1
branch binaries.
2. Extract it
3. CD inside the use case directory
4. Download the dataset (2G) just to be sure we have the same thing, and
place it in the working directory.
https://docs.google.com/uc?id=0BwZ_GPM33j6KV3ViNnpPcVFndUUexport=download
5. The first time, run ./test.sh. This will setup and run a test.
6. The next times, you can only run ./test --no-setup . This will only reset
the dataset as its initial state and re-run the test. You might have to run
the tests some times before experiencing it... but I'm always able with only
2-3 runs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8267) Only stream from unrepaired sstables during incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291696#comment-14291696 ] Marcus Eriksson commented on CASSANDRA-8267: the 2.1 patch is committed, so it will be in 2.1.3, the trunk-patch needs a new review, so i removed the 2.1.3 fix version so that this issue does not show up as a blocker for a 2.1.3 release - will re-add the version later Only stream from unrepaired sstables during incremental repair -- Key: CASSANDRA-8267 URL: https://issues.apache.org/jira/browse/CASSANDRA-8267 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 3.0 Attachments: 0001-Only-stream-from-unrepaired-sstables-during-incremen.patch, 8267-trunk.patch Seems we stream from all sstables even if we do incremental repair, we should limit this to only stream from the unrepaired sstables if we do incremental repair -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8267) Only stream from unrepaired sstables during incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291694#comment-14291694 ] Flavien Charlon commented on CASSANDRA-8267: Won't this be part of 2.1.3? I hope it will. Only stream from unrepaired sstables during incremental repair -- Key: CASSANDRA-8267 URL: https://issues.apache.org/jira/browse/CASSANDRA-8267 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 3.0 Attachments: 0001-Only-stream-from-unrepaired-sstables-during-incremen.patch, 8267-trunk.patch Seems we stream from all sstables even if we do incremental repair, we should limit this to only stream from the unrepaired sstables if we do incremental repair -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291760#comment-14291760
]
Maxim Podkolzine commented on CASSANDRA-8535:
-
Hi Joshua,
Yes, it looks better now. Before that I got the {{FileSystemException}} very
quickly, now I don't see it, at least after right from the start. I will keep
running to check it in long term.
There is a new one, that you might be interested in, since {{SSTableRewriter}}
was patched:
2015-01-15T16:23:47,760 [CompactionExecutor:6] ERROR
o.a.c.service.CassandraDaemon - Exception in thread
Thread[CompactionExecutor:6,1,main]
java.lang.AssertionError: null
at
org.apache.cassandra.io.sstable.IndexSummaryBuilder.build(IndexSummaryBuilder.java:133)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.IndexSummaryBuilder.build(IndexSummaryBuilder.java:128)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:476)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:459)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrowWindows(SSTableRewriter.java:394)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:353)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:341)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:321)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:213)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:76)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:227)
~[apache-cassandra-2.1.1.jar:2.1.2-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Also I see {{DecoderException}}s, which I previously assumed to be caused by
other errors, like one in this issue. Now it looks like a separate issue:
2015-01-15T16:03:25,613 [SharedPool-Worker-3] ERROR
o.apache.cassandra.transport.Message - Unexpected exception during request;
channel = [id: 0x9a614ce6, /127.0.0.1:51881 = /127.0.0.1:10030]
io.netty.handler.codec.DecoderException:
org.apache.cassandra.transport.ProtocolException: Invalid or unsupported
protocol version: 71
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:280)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[jira] [Commented] (CASSANDRA-8686) Introduce Latency Target for Stress
[ https://issues.apache.org/jira/browse/CASSANDRA-8686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292563#comment-14292563 ] Benedict commented on CASSANDRA-8686: - I like this idea. There's a lot of stress improvements needed in the near future, and I'll see if I can sneak this one in as well. Introduce Latency Target for Stress --- Key: CASSANDRA-8686 URL: https://issues.apache.org/jira/browse/CASSANDRA-8686 Project: Cassandra Issue Type: Improvement Reporter: jonathan lacefield Priority: Minor This item is a request to add a latency target to the rate option for the new stress tool. The goal of the latency target would be to provide a guideline for SLAs to the stress tool so the stress tool can determine threads and throughputs that can be sustained while meeting the SLA targets. For example: cassandra-stress [options/commands] -rate latency p90=5 p95=10 p99=100 The outcome of this command would be a stress execution that would gradually increase threads, and hence throughput (trans/sec), until the latency profile can no longer be satisfied with the current workload (yaml file definition) and/or cluster. This would provide a ceiling for throughput and connections for the given cluster, workload, and SLA profile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
[ https://issues.apache.org/jira/browse/CASSANDRA-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292316#comment-14292316 ] Ariel Weisberg edited comment on CASSANDRA-8684 at 1/26/15 11:09 PM: - !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1480884345format=image! !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1206341035format=image! The ever inscrutable results for OS X. I am very skeptical it is doing 13 gigabytes/second on a single core. I don't get why there is a gradual increase in speed as the size of the data being checksummed increases, and that speed up doesn't exist on Linux. Looking at a CPU monitor doesn't make it look like the application is using multiple cores. !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1911364989format=image! I think the real speed is 3 gigabytes/second which is what I have seen in the past and seen on Linux. There are faster hashes like xxhash or MurmurHash3 to consider that operate in the 5-6 gigabyte/second range. However a Java implementation of xxhash might not hit those numbers. The murmur3 implementation certainly doesn't. A native implementation incurs JNI overhead which is a factor for small values. was (Author: aweisberg): !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1480884345format=image! !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1206341035format=image! The ever inscrutable results for OS X. I am very skeptical it is doing 13 gigabytes/second on a single core. I don't get why there is a gradual increase in speed as the size of the data being checksummed increases, and that speed up doesn't exist on Linux. Looking at a CPU monitor doesn't make it look like the application is using multiple cores. !https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=1911364989format=image! I think the real speed is 3 gigabytes/second which is what I have seen in the past and seen on Linux. There are faster hashes like xxhash or MurmurHash3 to consider that operate in the 5-6 gigabyte/second range. However a Java implementation of xxhash might not hit those numbers. The murmur3 implementation certainly doesn't. A native implementation incurs JNI overhead and there is nothing packaged at the moment. Replace usage of Adler32 with CRC32 --- Key: CASSANDRA-8684 URL: https://issues.apache.org/jira/browse/CASSANDRA-8684 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg Attachments: CRCBenchmark.java, PureJavaCrc32.java, Sample.java I could not find a situation in which Adler32 outperformed PureJavaCrc32 much less the intrinsic from Java 8. For small allocations PureJavaCrc32 was much faster probably due to the JNI overhead of invoking the native Adler32 implementation where the array has to be allocated and copied. I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as well as a c3.8xlarge running Ubuntu 14.04. I think it makes sense to stop using Adler32 when generating new checksums. c3.8xlarge, results are time in milliseconds, lower is better ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|47636|46075|25782| |128|36755|36712|23782| |256|31194|32211|22731| |1024|27194|28792|22010| |1048576|25941|27807|21808| |536870912|25957|27840|21836| i5 ||Allocation size|Adler32|CRC32|PureJavaCrc32|| |64|50539|50466|26826| |128|37092|38533|24553| |256|30630|32938|23459| |1024|26064|29079|22592| |1048576|24357|27911|22481| |536870912|24838|28360|22853| Another fun fact. Performance of the CRC32 intrinsic appears to double from Sandy Bridge - Haswell. Unless I am measuring something different when going from Linux/Sandy to Haswell/OS X. The intrinsic/JDK 8 implementation also operates against DirectByteBuffers better and coding against the wrapper will get that boost when run with Java 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/2] cassandra git commit: Simplify logic of ABSC#BatchRemoveIterator#commit()
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 1bb0c149e - 33297bad0
Simplify logic of ABSC#BatchRemoveIterator#commit()
patch by Aleksey Yeschenko; reviewed by Joshua McKenzie for
CASSANDRA-8666
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b077cda8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b077cda8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b077cda8
Branch: refs/heads/cassandra-2.1
Commit: b077cda83bb1d32d5e6836027cfdfe3009b266b4
Parents: 2bf63f6
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jan 27 00:20:38 2015 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jan 27 00:20:38 2015 +0300
--
CHANGES.txt | 2 +-
.../cassandra/db/ArrayBackedSortedColumns.java | 53 +++-
2 files changed, 20 insertions(+), 35 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b077cda8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0d08cce..792f8c1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,6 @@
2.0.13:
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
- * Add batch remove iterator to ABSC (CASSANDRA-8414)
+ * Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b077cda8/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 8d553be..482f04f 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -314,43 +314,28 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
if (!removedAnything)
return;
-// the lowest index both not visited and known to be not
removed
-int keepIdx = removedIndexes.nextClearBit(0);
-// the running total of kept items
-int resultLength = 0;
-// start from the first not-removed cell, and shift left.
-int removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
-while (removeIdx = 0)
-{
-int length = removeIdx - keepIdx;
-if (length 0)
-{
-copy(keepIdx, resultLength, length);
-resultLength += length;
-}
-keepIdx = removedIndexes.nextClearBit(removeIdx + 1);
-if (keepIdx 0)
-keepIdx = columns.size();
-removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
-}
-// Copy everything after the last deleted column
-int length = columns.size() - keepIdx;
-if (length 0)
+int size = columns.size();
+int retainedCount = 0;
+int clearIdx, setIdx = -1;
+
+// shift all [clearIdx, setIdx) segments to the left, skipping
any removed columns
+while (true)
{
-copy(keepIdx, resultLength, length);
-resultLength += length;
+clearIdx = removedIndexes.nextClearBit(setIdx + 1);
+if (clearIdx = size)
+break; // nothing left to retain
+
+setIdx = removedIndexes.nextSetBit(clearIdx + 1);
+if (setIdx 0)
+setIdx = size; // no removals past retainIdx - copy
all remaining cells
+
+if (retainedCount != clearIdx)
+Collections.copy(columns.subList(retainedCount,
retainedCount + setIdx - clearIdx),
+ columns.subList(clearIdx, setIdx));
+retainedCount += (setIdx - clearIdx);
}
-columns.subList(resultLength, columns.size()).clear();
-}
-
-private void copy(int src, int dst, int len)
-{
-// [src, src+len) and [dst, dst+len) might overlap, but it's
okay because we're going from left to right
-assert dst = src : dst must not be greater than src;
-
-if (dst src)
-Collections.copy(columns.subList(dst, dst + len),
columns.subList(src, src + len));
+columns.subList(retainedCount, size).clear();
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0b08c994 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0b08c994 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0b08c994 Branch: refs/heads/trunk Commit: 0b08c994c199e372f8ac2cb7bf5fd3a216e14a25 Parents: 2e22bc9 33297ba Author: Aleksey Yeschenko alek...@apache.org Authored: Tue Jan 27 01:45:24 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Tue Jan 27 01:45:24 2015 +0300 -- CHANGES.txt | 2 +- .../cassandra/db/ArrayBackedSortedColumns.java | 54 +++- 2 files changed, 20 insertions(+), 36 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0b08c994/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0b08c994/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java --
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/33297bad Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/33297bad Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/33297bad Branch: refs/heads/trunk Commit: 33297bad040c660c8cd5ea6378189fffb100b0ab Parents: 1bb0c14 b077cda Author: Aleksey Yeschenko alek...@apache.org Authored: Tue Jan 27 01:44:54 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Tue Jan 27 01:44:54 2015 +0300 -- CHANGES.txt | 2 +- .../cassandra/db/ArrayBackedSortedColumns.java | 54 +++- 2 files changed, 20 insertions(+), 36 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/33297bad/CHANGES.txt -- diff --cc CHANGES.txt index 7673a3b,792f8c1..4d50496 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,77 -1,9 +1,77 @@@ -2.0.13: - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) +2.1.3 + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when generating error message + for some CQL syntax errors (CASSANDRA-8455) + * Scale memtable slab allocation logarithmically (CASSANDRA-7882) + * cassandra-stress simultaneous inserts over same seed (CASSANDRA-7964) + * Reduce cassandra-stress sampling memory requirements (CASSANDRA-7926) + * Ensure memtable flush cannot expire commit log entries from its future (CASSANDRA-8383) + * Make read defrag async to reclaim memtables (CASSANDRA-8459) + * Remove tmplink files for offline compactions (CASSANDRA-8321) + * Reduce maxHintsInProgress (CASSANDRA-8415) + * BTree updates may call provided update function twice (CASSANDRA-8018) + * Release sstable references after anticompaction
[1/3] cassandra git commit: Simplify logic of ABSC#BatchRemoveIterator#commit()
Repository: cassandra
Updated Branches:
refs/heads/trunk 2e22bc9aa - 0b08c994c
Simplify logic of ABSC#BatchRemoveIterator#commit()
patch by Aleksey Yeschenko; reviewed by Joshua McKenzie for
CASSANDRA-8666
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b077cda8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b077cda8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b077cda8
Branch: refs/heads/trunk
Commit: b077cda83bb1d32d5e6836027cfdfe3009b266b4
Parents: 2bf63f6
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jan 27 00:20:38 2015 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jan 27 00:20:38 2015 +0300
--
CHANGES.txt | 2 +-
.../cassandra/db/ArrayBackedSortedColumns.java | 53 +++-
2 files changed, 20 insertions(+), 35 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b077cda8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0d08cce..792f8c1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,6 @@
2.0.13:
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
- * Add batch remove iterator to ABSC (CASSANDRA-8414)
+ * Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b077cda8/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 8d553be..482f04f 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -314,43 +314,28 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
if (!removedAnything)
return;
-// the lowest index both not visited and known to be not
removed
-int keepIdx = removedIndexes.nextClearBit(0);
-// the running total of kept items
-int resultLength = 0;
-// start from the first not-removed cell, and shift left.
-int removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
-while (removeIdx = 0)
-{
-int length = removeIdx - keepIdx;
-if (length 0)
-{
-copy(keepIdx, resultLength, length);
-resultLength += length;
-}
-keepIdx = removedIndexes.nextClearBit(removeIdx + 1);
-if (keepIdx 0)
-keepIdx = columns.size();
-removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
-}
-// Copy everything after the last deleted column
-int length = columns.size() - keepIdx;
-if (length 0)
+int size = columns.size();
+int retainedCount = 0;
+int clearIdx, setIdx = -1;
+
+// shift all [clearIdx, setIdx) segments to the left, skipping
any removed columns
+while (true)
{
-copy(keepIdx, resultLength, length);
-resultLength += length;
+clearIdx = removedIndexes.nextClearBit(setIdx + 1);
+if (clearIdx = size)
+break; // nothing left to retain
+
+setIdx = removedIndexes.nextSetBit(clearIdx + 1);
+if (setIdx 0)
+setIdx = size; // no removals past retainIdx - copy
all remaining cells
+
+if (retainedCount != clearIdx)
+Collections.copy(columns.subList(retainedCount,
retainedCount + setIdx - clearIdx),
+ columns.subList(clearIdx, setIdx));
+retainedCount += (setIdx - clearIdx);
}
-columns.subList(resultLength, columns.size()).clear();
-}
-
-private void copy(int src, int dst, int len)
-{
-// [src, src+len) and [dst, dst+len) might overlap, but it's
okay because we're going from left to right
-assert dst = src : dst must not be greater than src;
-
-if (dst src)
-Collections.copy(columns.subList(dst, dst + len),
columns.subList(src, src + len));
+columns.subList(retainedCount, size).clear();
}
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/33297bad Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/33297bad Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/33297bad Branch: refs/heads/cassandra-2.1 Commit: 33297bad040c660c8cd5ea6378189fffb100b0ab Parents: 1bb0c14 b077cda Author: Aleksey Yeschenko alek...@apache.org Authored: Tue Jan 27 01:44:54 2015 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Tue Jan 27 01:44:54 2015 +0300 -- CHANGES.txt | 2 +- .../cassandra/db/ArrayBackedSortedColumns.java | 54 +++- 2 files changed, 20 insertions(+), 36 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/33297bad/CHANGES.txt -- diff --cc CHANGES.txt index 7673a3b,792f8c1..4d50496 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,77 -1,9 +1,77 @@@ -2.0.13: - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) +2.1.3 + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when generating error message + for some CQL syntax errors (CASSANDRA-8455) + * Scale memtable slab allocation logarithmically (CASSANDRA-7882) + * cassandra-stress simultaneous inserts over same seed (CASSANDRA-7964) + * Reduce cassandra-stress sampling memory requirements (CASSANDRA-7926) + * Ensure memtable flush cannot expire commit log entries from its future (CASSANDRA-8383) + * Make read defrag async to reclaim memtables (CASSANDRA-8459) + * Remove tmplink files for offline compactions (CASSANDRA-8321) + * Reduce maxHintsInProgress (CASSANDRA-8415) + * BTree updates may call provided update function twice (CASSANDRA-8018) + * Release sstable references after
[jira] [Commented] (CASSANDRA-7933) Update cassandra-stress README
[ https://issues.apache.org/jira/browse/CASSANDRA-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292678#comment-14292678 ] Benedict commented on CASSANDRA-7933: - Yes. I have a few minor commits to make before then, and this is amongst them. Update cassandra-stress README -- Key: CASSANDRA-7933 URL: https://issues.apache.org/jira/browse/CASSANDRA-7933 Project: Cassandra Issue Type: Task Reporter: Benedict Assignee: Philip Thompson Priority: Minor Fix For: 2.1.3 Attachments: 7933-2.txt, CASSANDRA-7933.txt There is a README in the tools/stress directory. It is completely out of date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292404#comment-14292404
]
Mark Cooper commented on CASSANDRA-8061:
[~JoshuaMcKenzie] - oops yes, meant to say I am now running this patch.
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Index.db
113M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Data.db
116M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-197-Data.db
712K
[jira] [Commented] (CASSANDRA-8061) tmplink files are not removed
[
https://issues.apache.org/jira/browse/CASSANDRA-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292432#comment-14292432
]
Benedict commented on CASSANDRA-8061:
-
Looks like a different bug, introduced by CASSANDRA-8124, that only presents
during failure, and overrides the causative exception with one during cleanup.
This may possibly be the failure we're really looking for. Or it may be another
new bug - this is the risk we run when testing out mid release :)
I will upload a fix directly to the branch for this bug, and also an
improvement to ensure such bugs in future don't mask the underlying exception
tmplink files are not removed
-
Key: CASSANDRA-8061
URL: https://issues.apache.org/jira/browse/CASSANDRA-8061
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: Gianluca Borello
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: 8061_v1.txt, 8248-thread_dump.txt
After installing 2.1.0, I'm experiencing a bunch of tmplink files that are
filling my disk. I found https://issues.apache.org/jira/browse/CASSANDRA-7803
and that is very similar, and I confirm it happens both on 2.1.0 as well as
from the latest commit on the cassandra-2.1 branch
(https://github.com/apache/cassandra/commit/aca80da38c3d86a40cc63d9a122f7d45258e4685
from the cassandra-2.1)
Even starting with a clean keyspace, after a few hours I get:
{noformat}
$ sudo find /raid0 | grep tmplink | xargs du -hs
2.7G
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Data.db
13M
/raid0/cassandra/data/draios/protobuf1-ccc6dce04beb11e4abf997b38fbf920b/draios-protobuf1-tmplink-ka-4515-Index.db
1.8G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Data.db
12M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-1788-Index.db
5.2M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Index.db
822M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-2678-Data.db
7.3M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Index.db
1.2G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3283-Data.db
6.7M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Index.db
1.1G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-3951-Data.db
11M
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Index.db
1.7G
/raid0/cassandra/data/draios/protobuf_by_agent1-cd071a304beb11e4abf997b38fbf920b/draios-protobuf_by_agent1-tmplink-ka-4799-Data.db
812K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Index.db
122M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-208-Data.db
744K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-739-Index.db
660K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-193-Index.db
796K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Index.db
137M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-230-Data.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Data.db
139M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-234-Data.db
940K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Index.db
936K
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-269-Index.db
161M
/raid0/cassandra/data/draios/mounted_fs_by_agent1-d7bf3e304beb11e4abf997b38fbf920b/draios-mounted_fs_by_agent1-tmplink-ka-786-Data.db
672K
cassandra git commit: Simplify logic of ABSC#BatchRemoveIterator#commit()
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 2bf63f61e - b077cda83
Simplify logic of ABSC#BatchRemoveIterator#commit()
patch by Aleksey Yeschenko; reviewed by Joshua McKenzie for
CASSANDRA-8666
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b077cda8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b077cda8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b077cda8
Branch: refs/heads/cassandra-2.0
Commit: b077cda83bb1d32d5e6836027cfdfe3009b266b4
Parents: 2bf63f6
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Jan 27 00:20:38 2015 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Jan 27 00:20:38 2015 +0300
--
CHANGES.txt | 2 +-
.../cassandra/db/ArrayBackedSortedColumns.java | 53 +++-
2 files changed, 20 insertions(+), 35 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b077cda8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0d08cce..792f8c1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,6 @@
2.0.13:
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
- * Add batch remove iterator to ABSC (CASSANDRA-8414)
+ * Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b077cda8/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 8d553be..482f04f 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -314,43 +314,28 @@ public class ArrayBackedSortedColumns extends
AbstractThreadUnsafeSortedColumns
if (!removedAnything)
return;
-// the lowest index both not visited and known to be not
removed
-int keepIdx = removedIndexes.nextClearBit(0);
-// the running total of kept items
-int resultLength = 0;
-// start from the first not-removed cell, and shift left.
-int removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
-while (removeIdx = 0)
-{
-int length = removeIdx - keepIdx;
-if (length 0)
-{
-copy(keepIdx, resultLength, length);
-resultLength += length;
-}
-keepIdx = removedIndexes.nextClearBit(removeIdx + 1);
-if (keepIdx 0)
-keepIdx = columns.size();
-removeIdx = removedIndexes.nextSetBit(keepIdx + 1);
-}
-// Copy everything after the last deleted column
-int length = columns.size() - keepIdx;
-if (length 0)
+int size = columns.size();
+int retainedCount = 0;
+int clearIdx, setIdx = -1;
+
+// shift all [clearIdx, setIdx) segments to the left, skipping
any removed columns
+while (true)
{
-copy(keepIdx, resultLength, length);
-resultLength += length;
+clearIdx = removedIndexes.nextClearBit(setIdx + 1);
+if (clearIdx = size)
+break; // nothing left to retain
+
+setIdx = removedIndexes.nextSetBit(clearIdx + 1);
+if (setIdx 0)
+setIdx = size; // no removals past retainIdx - copy
all remaining cells
+
+if (retainedCount != clearIdx)
+Collections.copy(columns.subList(retainedCount,
retainedCount + setIdx - clearIdx),
+ columns.subList(clearIdx, setIdx));
+retainedCount += (setIdx - clearIdx);
}
-columns.subList(resultLength, columns.size()).clear();
-}
-
-private void copy(int src, int dst, int len)
-{
-// [src, src+len) and [dst, dst+len) might overlap, but it's
okay because we're going from left to right
-assert dst = src : dst must not be greater than src;
-
-if (dst src)
-Collections.copy(columns.subList(dst, dst + len),
columns.subList(src, src + len));
+columns.subList(retainedCount, size).clear();
