[jira] [Commented] (CASSANDRA-6450) sstable2json hangs if keyspace uses authentication
[ https://issues.apache.org/jira/browse/CASSANDRA-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855447#comment-13855447 ] Daniel Kador commented on CASSANDRA-6450: - Just wanted to update this to say that this is blocking us from doing some investigation into whether or not we're getting hit by a tombstone. Would love to hear if this shouldn't work and we're using the tool wrong or if it should and it's a bug. Thanks! > sstable2json hangs if keyspace uses authentication > -- > > Key: CASSANDRA-6450 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6450 > Project: Cassandra > Issue Type: Bug > Environment: Ubuntu 12 >Reporter: Josh Dzielak >Priority: Minor > > Running sstable2json against an authenticated keyspace hangs indefinitely. > True for other utilities based on SSTableExport as well. > Running sstable2json against other unauthenticated keyspaces in the same > node/cluster was successful. Running against any CF in the keyspace with > password authentication on resulted in a hang. > It looks like it gets about to: > Table table = Table.open(descriptor.ksname); or > table.getColumnFamilyStore(baseName); > in SSTableExport.java but no farther. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6420) CassandraStorage should not assume all DataBags are DefaultDataBags
[ https://issues.apache.org/jira/browse/CASSANDRA-6420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855402#comment-13855402 ] Mike Spertus commented on CASSANDRA-6420: - Thanks, Brandon! > CassandraStorage should not assume all DataBags are DefaultDataBags > --- > > Key: CASSANDRA-6420 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6420 > Project: Cassandra > Issue Type: Bug > Components: Hadoop > Environment: All environments >Reporter: Mike Spertus >Assignee: Mike Spertus > Labels: pig > Fix For: 1.2.14, 2.0.4 > > Attachments: patch.txt > > Original Estimate: 1h > Remaining Estimate: 1h > > CassandraStorage improperly assumes all DataBags are DefaultDataBags. As a > result, natural Pig code can't be used with CassandraStorage. For example: > {quote} > {{B = FOREACH A GENERATE $0, TOBAG(TOTUPLE($1, $2));}} > {{STORE B into 'cassandra://MyKeySpace/MyColumnFamily' using > CassandraStorage();}} > {quote} > fails with a complaint that a {{NonSpillableDataBag}} can't be converted into > a {{DefaultDataBag}}. > Since the {{CassandraStorage}} code only calls methods from {{DataBag}}, > there is no need for this artifical restriction. After applying the attached > patch, the above code works fine, making CassandraStorage much easier to use. > This is my first submission to Cassandra, so I apologize for any incorrect > process. Please let me know what I should do differently. In particular, I am > a little unclear where I should put the test. I am thinking I should put it > in ThriftColumnFamilyTest.java. Is this correct or should it be somewhere > else? I'll create a test as soon as I understand. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6522) DroppableTombstoneRatio JMX value is 0.0 for all CFs
Daniel Kador created CASSANDRA-6522: --- Summary: DroppableTombstoneRatio JMX value is 0.0 for all CFs Key: CASSANDRA-6522 URL: https://issues.apache.org/jira/browse/CASSANDRA-6522 Project: Cassandra Issue Type: Bug Components: Core Environment: Ubuntu 12.04 LTS, Cassandra 1.2.8 Reporter: Daniel Kador Priority: Minor We're seeing that the JMX value for DroppableTombstoneRatio for all our CFs is 0.0. On the face of it that seems wrong since we've definitely issued a ton of deletes for row keys to expire some old data that we no longer need (and it definitely hasn't been reclaimed from disk yet). Am I misunderstanding what this means / how to use it? We're on 1.2.8 and using leveled compaction for all our CFs. gc_grace_seconds is set to 1 day and we've issued a series of deletes over a day ago, so gc_grace has elapsed. Cluster is 18 nodes. Two DCs, so 9 nodes in each DC. Each node has capacity for 1.5TB or so and is sitting with about 1TB under management. That's why we wanted to do deletes, obviously. Most of that 1TB is a single CF (called "events") which represents intermediate state for us that we can delete. Happy to provide any more info, just let me know. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855248#comment-13855248 ] Daniel Kador commented on CASSANDRA-5159: - Filed CASSANDRA-6522. Thanks! > Expose droppable tombstone ratio stats over JMX > --- > > Key: CASSANDRA-5159 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.2.0 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.2.1 > > Attachments: > 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch > > > Could be nice to be able to jmx-monitor the tombstone histogram collected > during compactions to see how many gc-able tombstones there are left due to > the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855247#comment-13855247 ] Marcus Eriksson commented on CASSANDRA-5159: i have checked this attribute on clusters and it has worked then (getting reasonable values), versions probably a bit later than yours (1.2.11) if you could describe your config etc in a new ticket i could look in to it a bit closer. > Expose droppable tombstone ratio stats over JMX > --- > > Key: CASSANDRA-5159 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.2.0 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.2.1 > > Attachments: > 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch > > > Could be nice to be able to jmx-monitor the tombstone histogram collected > during compactions to see how many gc-able tombstones there are left due to > the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6236) Update native protocol server to Netty 4
[ https://issues.apache.org/jira/browse/CASSANDRA-6236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855229#comment-13855229 ] Aleksey Yeschenko commented on CASSANDRA-6236: -- Or maybe skip Netty 4 and go directly to Netty 5 - http://netty.io/wiki/new-and-noteworthy-in-5.x.html > Update native protocol server to Netty 4 > > > Key: CASSANDRA-6236 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6236 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Priority: Minor > Fix For: 2.1 > > > We should switch to Netty 4 at some point, since it's the future. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4206) AssertionError: originally calculated column size of 629444349 but now it is 588008950
[ https://issues.apache.org/jira/browse/CASSANDRA-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855228#comment-13855228 ] Daniel Kador commented on CASSANDRA-4206: - We got desperate and tried multithreaded_compaction=false yesterday afternoon and it seems to have worked for us. Pending compactions across the cluster have dropped from over 20,000 to under 600 (we're not quite done yet). The numbers were so high because we added six new nodes and they were unable to compact quickly until we made this settings change. No idea why it worked for us but thought I'd share our experience. If there's any other useful information I can share, let me know. > AssertionError: originally calculated column size of 629444349 but now it is > 588008950 > -- > > Key: CASSANDRA-4206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4206 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.0.9 > Environment: Debian Squeeze Linux, kernel 2.6.32, sun-java6-bin > 6.26-0squeeze1 >Reporter: Patrik Modesto > > I've 4 node cluster of Cassandra 1.0.9. There is a rfTest3 keyspace with RF=3 > and one CF with two secondary indexes. I'm importing data into this CF using > Hadoop Mapreduce job, each row has less than 10 colkumns. From JMX: > MaxRowSize: 1597 > MeanRowSize: 369 > And there are some tens of millions of rows. > It's write-heavy usage and there is a big pressure on each node, there are > quite some dropped mutations on each node. After ~12 hours of inserting I see > these assertion exceptiona on 3 out of four nodes: > {noformat} > ERROR 06:25:40,124 Fatal exception in thread Thread[HintedHandoff:1,1,main] > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.AssertionError: originally calculated column size of 629444349 but > now it is 588008950 >at > org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:388) >at > org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:256) >at > org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:84) >at > org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:437) >at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) >at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >at java.lang.Thread.run(Thread.java:662) > Caused by: java.util.concurrent.ExecutionException: > java.lang.AssertionError: originally calculated column size of > 629444349 but now it is 588008950 >at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) >at java.util.concurrent.FutureTask.get(FutureTask.java:83) >at > org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:384) >... 7 more > Caused by: java.lang.AssertionError: originally calculated column size > of 629444349 but now it is 588008950 >at > org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124) >at > org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160) >at > org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:161) >at > org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:380) >at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >at java.util.concurrent.FutureTask.run(FutureTask.java:138) >... 3 more > {noformat} > Few lines regarding Hints from the output.log: > {noformat} > INFO 06:21:26,202 Compacting large row > system/HintsColumnFamily:7000 (1712834057 bytes) > incrementally > INFO 06:22:52,610 Compacting large row > system/HintsColumnFamily:1000 (2616073981 bytes) > incrementally > INFO 06:22:59,111 flushing high-traffic column family CFS(Keyspace='system', > ColumnFamily='HintsColumnFamily') (estimated 305147360 bytes) > INFO 06:22:59,813 Enqueuing flush of > Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes, > 7452 ops) > INFO 06:22:59,814 Writing > Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes, > 7452 ops) > {noformat} > I think the problem may be somehow connected to an IntegerType secondary > index. I had a different problem with CF with two secondary indexes, the > first UTF8Type, the second IntegerType. After a few hours of inserting data > in
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855223#comment-13855223 ] Daniel Kador commented on CASSANDRA-5159: - Yes, gc_grace is currently set to 1 day and we've waited that long. > Expose droppable tombstone ratio stats over JMX > --- > > Key: CASSANDRA-5159 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.2.0 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.2.1 > > Attachments: > 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch > > > Could be nice to be able to jmx-monitor the tombstone histogram collected > during compactions to see how many gc-able tombstones there are left due to > the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
[ https://issues.apache.org/jira/browse/CASSANDRA-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-6521. -- Resolution: Fixed Committed, thanks, Ben. > Thrift should validate SliceRange start and finish lengths > -- > > Key: CASSANDRA-6521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 > Project: Cassandra > Issue Type: Bug >Reporter: Ben Bromhead >Assignee: Ben Bromhead > Fix For: 1.2.14, 2.0.4 > > Attachments: 6521.txt > > > To quote [~benbromhead]: > bq. It appears that Cassandra does not check the length of a column name that > is part of a range predicate for a *_slice query before it serialises the > slice query to pass to the replicas. Names with a length greater than 0x > cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. > This further causes subsequent reads on the node to fail until Cassandra is > restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[4/4] git commit: Fix merge
Fix merge Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/34235ad7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/34235ad7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/34235ad7 Branch: refs/heads/trunk Commit: 34235ad7b5722a5a5ee9680ba3a2d563a67ecc6f Parents: 85863e2 Author: Aleksey Yeschenko Authored: Sun Dec 22 20:10:59 2013 +0300 Committer: Aleksey Yeschenko Committed: Sun Dec 22 20:10:59 2013 +0300 -- src/java/org/apache/cassandra/thrift/ThriftValidation.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/34235ad7/src/java/org/apache/cassandra/thrift/ThriftValidation.java -- diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java b/src/java/org/apache/cassandra/thrift/ThriftValidation.java index b597203..d491636 100644 --- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java +++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java @@ -268,7 +268,7 @@ public class ThriftValidation if (range.count < 0) throw new org.apache.cassandra.exceptions.InvalidRequestException("get_slice requires non-negative count"); -int maxNameLength = org.apache.cassandra.db.Column.MAX_NAME_LENGTH; +int maxNameLength = Cell.MAX_NAME_LENGTH; if (range.start.remaining() > maxNameLength) throw new org.apache.cassandra.exceptions.InvalidRequestException("range start length cannot be larger than " + maxNameLength); if (range.finish.remaining() > maxNameLength)
[3/4] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85863e27 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85863e27 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85863e27 Branch: refs/heads/trunk Commit: 85863e273ecc5808fc9085b0f8f63aa02c4be456 Parents: d0ab751 03f4c92 Author: Aleksey Yeschenko Authored: Sun Dec 22 20:09:57 2013 +0300 Committer: Aleksey Yeschenko Committed: Sun Dec 22 20:09:57 2013 +0300 -- CHANGES.txt | 1 + .../org/apache/cassandra/thrift/ThriftValidation.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/85863e27/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/85863e27/src/java/org/apache/cassandra/thrift/ThriftValidation.java --
[1/4] git commit: Validate SliceRange start and finish lengths
Updated Branches: refs/heads/trunk d0ab7519c -> 34235ad7b Validate SliceRange start and finish lengths patch by Ben Bromhead; reviewed by Aleksey Yeschenko for CASSANDRA-6521 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a58a95dd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a58a95dd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a58a95dd Branch: refs/heads/trunk Commit: a58a95ddbb438cf8a2980d71b5cdbc2e63642685 Parents: 9244923 Author: Aleksey Yeschenko Authored: Sun Dec 22 20:03:05 2013 +0300 Committer: Aleksey Yeschenko Committed: Sun Dec 22 20:03:05 2013 +0300 -- CHANGES.txt | 1 + .../org/apache/cassandra/thrift/ThriftValidation.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6f6c131..4eee4ee 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -4,6 +4,7 @@ * Don't resubmit counter mutation runnables internally (CASSANDRA-6427) * Don't drop local mutations without a trace (CASSANDRA-6510) * Don't allow null max_hint_window_in_ms (CASSANDRA-6419) + * Validate SliceRange start and finish lengths (CASSANDRA-6521) 1.2.13 http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/src/java/org/apache/cassandra/thrift/ThriftValidation.java -- diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java b/src/java/org/apache/cassandra/thrift/ThriftValidation.java index 5882e27..442ea4e 100644 --- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java +++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java @@ -228,6 +228,15 @@ public class ThriftValidation public static void validateRange(CFMetaData metadata, ColumnParent column_parent, SliceRange range) throws org.apache.cassandra.exceptions.InvalidRequestException { +if (range.count < 0) +throw new org.apache.cassandra.exceptions.InvalidRequestException("get_slice requires non-negative count"); + +if (range.start.remaining() > IColumn.MAX_NAME_LENGTH) +throw new org.apache.cassandra.exceptions.InvalidRequestException("range start length cannot be larger than " + IColumn.MAX_NAME_LENGTH); + +if (range.finish.remaining() > IColumn.MAX_NAME_LENGTH) +throw new org.apache.cassandra.exceptions.InvalidRequestException("range finish length cannot be larger than " + IColumn.MAX_NAME_LENGTH); + AbstractType comparator = metadata.getComparatorFor(column_parent.super_column); try { @@ -239,9 +248,6 @@ public class ThriftValidation throw new org.apache.cassandra.exceptions.InvalidRequestException(e.getMessage()); } -if (range.count < 0) -throw new org.apache.cassandra.exceptions.InvalidRequestException("get_slice requires non-negative count"); - Comparator orderedComparator = range.isReversed() ? comparator.reverseComparator : comparator; if (range.start.remaining() > 0 && range.finish.remaining() > 0
[2/4] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/thrift/ThriftValidation.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03f4c92d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03f4c92d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03f4c92d Branch: refs/heads/trunk Commit: 03f4c92d14ec7bbdac7524f531423c79015c8061 Parents: b5c9b49 a58a95d Author: Aleksey Yeschenko Authored: Sun Dec 22 20:09:18 2013 +0300 Committer: Aleksey Yeschenko Committed: Sun Dec 22 20:09:18 2013 +0300 -- CHANGES.txt | 1 + .../org/apache/cassandra/thrift/ThriftValidation.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/CHANGES.txt -- diff --cc CHANGES.txt index b80c821,4eee4ee..f38b58f --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -34,50 -28,9 +34,51 @@@ Merged from 1.2 (CASSANDRA-6413) * (Hadoop) add describe_local_ring (CASSANDRA-6268) * Fix handling of concurrent directory creation failure (CASSANDRA-6459) + * Allow executing CREATE statements multiple times (CASSANDRA-6471) + * Don't send confusing info with timeouts (CASSANDRA-6491) + * Don't resubmit counter mutation runnables internally (CASSANDRA-6427) + * Don't drop local mutations without a trace (CASSANDRA-6510) + * Don't allow null max_hint_window_in_ms (CASSANDRA-6419) ++ * Validate SliceRange start and finish lengths (CASSANDRA-6521) -1.2.12 +2.0.3 + * Fix FD leak on slice read path (CASSANDRA-6275) + * Cancel read meter task when closing SSTR (CASSANDRA-6358) + * free off-heap IndexSummary during bulk (CASSANDRA-6359) + * Recover from IOException in accept() thread (CASSANDRA-6349) + * Improve Gossip tolerance of abnormally slow tasks (CASSANDRA-6338) + * Fix trying to hint timed out counter writes (CASSANDRA-6322) + * Allow restoring specific columnfamilies from archived CL (CASSANDRA-4809) + * Avoid flushing compaction_history after each operation (CASSANDRA-6287) + * Fix repair assertion error when tombstones expire (CASSANDRA-6277) + * Skip loading corrupt key cache (CASSANDRA-6260) + * Fixes for compacting larger-than-memory rows (CASSANDRA-6274) + * Compact hottest sstables first and optionally omit coldest from + compaction entirely (CASSANDRA-6109) + * Fix modifying column_metadata from thrift (CASSANDRA-6182) + * cqlsh: fix LIST USERS output (CASSANDRA-6242) + * Add IRequestSink interface (CASSANDRA-6248) + * Update memtable size while flushing (CASSANDRA-6249) + * Provide hooks around CQL2/CQL3 statement execution (CASSANDRA-6252) + * Require Permission.SELECT for CAS updates (CASSANDRA-6247) + * New CQL-aware SSTableWriter (CASSANDRA-5894) + * Reject CAS operation when the protocol v1 is used (CASSANDRA-6270) + * Correctly throw error when frame too large (CASSANDRA-5981) + * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299) + * Fix CQL3 table validation in Thrift (CASSANDRA-6140) + * Fix bug missing results with IN clauses (CASSANDRA-6327) + * Fix paging with reversed slices (CASSANDRA-6343) + * Set minTimestamp correctly to be able to drop expired sstables (CASSANDRA-6337) + * Support NaN and Infinity as float literals (CASSANDRA-6003) + * Remove RF from nodetool ring output (CASSANDRA-6289) + * Fix attempting to flush empty rows (CASSANDRA-6374) + * Fix potential out of bounds exception when paging (CASSANDRA-6333) +Merged from 1.2: + * Optimize FD phi calculation (CASSANDRA-6386) + * Improve initial FD phi estimate when starting up (CASSANDRA-6385) + * Don't list CQL3 table in CLI describe even if named explicitely + (CASSANDRA-5750) * Invalidate row cache when dropping CF (CASSANDRA-6351) * add non-jamm path for cached statements (CASSANDRA-6293) * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/src/java/org/apache/cassandra/thrift/ThriftValidation.java -- diff --cc src/java/org/apache/cassandra/thrift/ThriftValidation.java index 3a831f4,442ea4e..f2efc03 --- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java +++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java @@@ -268,7 -228,16 +268,16 @@@ public class ThriftValidatio public static void validateRange(CFMetaData metadata, ColumnParent column_parent, SliceRange range) throws org.apache.cassandra.exceptions.InvalidRequestException { + if (range.count < 0) + throw new org.apache.cassandra.e
[2/2] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/thrift/ThriftValidation.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03f4c92d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03f4c92d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03f4c92d Branch: refs/heads/cassandra-2.0 Commit: 03f4c92d14ec7bbdac7524f531423c79015c8061 Parents: b5c9b49 a58a95d Author: Aleksey Yeschenko Authored: Sun Dec 22 20:09:18 2013 +0300 Committer: Aleksey Yeschenko Committed: Sun Dec 22 20:09:18 2013 +0300 -- CHANGES.txt | 1 + .../org/apache/cassandra/thrift/ThriftValidation.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/CHANGES.txt -- diff --cc CHANGES.txt index b80c821,4eee4ee..f38b58f --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -34,50 -28,9 +34,51 @@@ Merged from 1.2 (CASSANDRA-6413) * (Hadoop) add describe_local_ring (CASSANDRA-6268) * Fix handling of concurrent directory creation failure (CASSANDRA-6459) + * Allow executing CREATE statements multiple times (CASSANDRA-6471) + * Don't send confusing info with timeouts (CASSANDRA-6491) + * Don't resubmit counter mutation runnables internally (CASSANDRA-6427) + * Don't drop local mutations without a trace (CASSANDRA-6510) + * Don't allow null max_hint_window_in_ms (CASSANDRA-6419) ++ * Validate SliceRange start and finish lengths (CASSANDRA-6521) -1.2.12 +2.0.3 + * Fix FD leak on slice read path (CASSANDRA-6275) + * Cancel read meter task when closing SSTR (CASSANDRA-6358) + * free off-heap IndexSummary during bulk (CASSANDRA-6359) + * Recover from IOException in accept() thread (CASSANDRA-6349) + * Improve Gossip tolerance of abnormally slow tasks (CASSANDRA-6338) + * Fix trying to hint timed out counter writes (CASSANDRA-6322) + * Allow restoring specific columnfamilies from archived CL (CASSANDRA-4809) + * Avoid flushing compaction_history after each operation (CASSANDRA-6287) + * Fix repair assertion error when tombstones expire (CASSANDRA-6277) + * Skip loading corrupt key cache (CASSANDRA-6260) + * Fixes for compacting larger-than-memory rows (CASSANDRA-6274) + * Compact hottest sstables first and optionally omit coldest from + compaction entirely (CASSANDRA-6109) + * Fix modifying column_metadata from thrift (CASSANDRA-6182) + * cqlsh: fix LIST USERS output (CASSANDRA-6242) + * Add IRequestSink interface (CASSANDRA-6248) + * Update memtable size while flushing (CASSANDRA-6249) + * Provide hooks around CQL2/CQL3 statement execution (CASSANDRA-6252) + * Require Permission.SELECT for CAS updates (CASSANDRA-6247) + * New CQL-aware SSTableWriter (CASSANDRA-5894) + * Reject CAS operation when the protocol v1 is used (CASSANDRA-6270) + * Correctly throw error when frame too large (CASSANDRA-5981) + * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299) + * Fix CQL3 table validation in Thrift (CASSANDRA-6140) + * Fix bug missing results with IN clauses (CASSANDRA-6327) + * Fix paging with reversed slices (CASSANDRA-6343) + * Set minTimestamp correctly to be able to drop expired sstables (CASSANDRA-6337) + * Support NaN and Infinity as float literals (CASSANDRA-6003) + * Remove RF from nodetool ring output (CASSANDRA-6289) + * Fix attempting to flush empty rows (CASSANDRA-6374) + * Fix potential out of bounds exception when paging (CASSANDRA-6333) +Merged from 1.2: + * Optimize FD phi calculation (CASSANDRA-6386) + * Improve initial FD phi estimate when starting up (CASSANDRA-6385) + * Don't list CQL3 table in CLI describe even if named explicitely + (CASSANDRA-5750) * Invalidate row cache when dropping CF (CASSANDRA-6351) * add non-jamm path for cached statements (CASSANDRA-6293) * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f4c92d/src/java/org/apache/cassandra/thrift/ThriftValidation.java -- diff --cc src/java/org/apache/cassandra/thrift/ThriftValidation.java index 3a831f4,442ea4e..f2efc03 --- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java +++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java @@@ -268,7 -228,16 +268,16 @@@ public class ThriftValidatio public static void validateRange(CFMetaData metadata, ColumnParent column_parent, SliceRange range) throws org.apache.cassandra.exceptions.InvalidRequestException { + if (range.count < 0) + throw new org.apache.cas
[1/2] git commit: Validate SliceRange start and finish lengths
Updated Branches: refs/heads/cassandra-2.0 b5c9b498f -> 03f4c92d1 Validate SliceRange start and finish lengths patch by Ben Bromhead; reviewed by Aleksey Yeschenko for CASSANDRA-6521 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a58a95dd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a58a95dd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a58a95dd Branch: refs/heads/cassandra-2.0 Commit: a58a95ddbb438cf8a2980d71b5cdbc2e63642685 Parents: 9244923 Author: Aleksey Yeschenko Authored: Sun Dec 22 20:03:05 2013 +0300 Committer: Aleksey Yeschenko Committed: Sun Dec 22 20:03:05 2013 +0300 -- CHANGES.txt | 1 + .../org/apache/cassandra/thrift/ThriftValidation.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6f6c131..4eee4ee 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -4,6 +4,7 @@ * Don't resubmit counter mutation runnables internally (CASSANDRA-6427) * Don't drop local mutations without a trace (CASSANDRA-6510) * Don't allow null max_hint_window_in_ms (CASSANDRA-6419) + * Validate SliceRange start and finish lengths (CASSANDRA-6521) 1.2.13 http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/src/java/org/apache/cassandra/thrift/ThriftValidation.java -- diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java b/src/java/org/apache/cassandra/thrift/ThriftValidation.java index 5882e27..442ea4e 100644 --- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java +++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java @@ -228,6 +228,15 @@ public class ThriftValidation public static void validateRange(CFMetaData metadata, ColumnParent column_parent, SliceRange range) throws org.apache.cassandra.exceptions.InvalidRequestException { +if (range.count < 0) +throw new org.apache.cassandra.exceptions.InvalidRequestException("get_slice requires non-negative count"); + +if (range.start.remaining() > IColumn.MAX_NAME_LENGTH) +throw new org.apache.cassandra.exceptions.InvalidRequestException("range start length cannot be larger than " + IColumn.MAX_NAME_LENGTH); + +if (range.finish.remaining() > IColumn.MAX_NAME_LENGTH) +throw new org.apache.cassandra.exceptions.InvalidRequestException("range finish length cannot be larger than " + IColumn.MAX_NAME_LENGTH); + AbstractType comparator = metadata.getComparatorFor(column_parent.super_column); try { @@ -239,9 +248,6 @@ public class ThriftValidation throw new org.apache.cassandra.exceptions.InvalidRequestException(e.getMessage()); } -if (range.count < 0) -throw new org.apache.cassandra.exceptions.InvalidRequestException("get_slice requires non-negative count"); - Comparator orderedComparator = range.isReversed() ? comparator.reverseComparator : comparator; if (range.start.remaining() > 0 && range.finish.remaining() > 0
git commit: Validate SliceRange start and finish lengths
Updated Branches: refs/heads/cassandra-1.2 924492345 -> a58a95ddb Validate SliceRange start and finish lengths patch by Ben Bromhead; reviewed by Aleksey Yeschenko for CASSANDRA-6521 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a58a95dd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a58a95dd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a58a95dd Branch: refs/heads/cassandra-1.2 Commit: a58a95ddbb438cf8a2980d71b5cdbc2e63642685 Parents: 9244923 Author: Aleksey Yeschenko Authored: Sun Dec 22 20:03:05 2013 +0300 Committer: Aleksey Yeschenko Committed: Sun Dec 22 20:03:05 2013 +0300 -- CHANGES.txt | 1 + .../org/apache/cassandra/thrift/ThriftValidation.java | 12 +--- 2 files changed, 10 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6f6c131..4eee4ee 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -4,6 +4,7 @@ * Don't resubmit counter mutation runnables internally (CASSANDRA-6427) * Don't drop local mutations without a trace (CASSANDRA-6510) * Don't allow null max_hint_window_in_ms (CASSANDRA-6419) + * Validate SliceRange start and finish lengths (CASSANDRA-6521) 1.2.13 http://git-wip-us.apache.org/repos/asf/cassandra/blob/a58a95dd/src/java/org/apache/cassandra/thrift/ThriftValidation.java -- diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java b/src/java/org/apache/cassandra/thrift/ThriftValidation.java index 5882e27..442ea4e 100644 --- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java +++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java @@ -228,6 +228,15 @@ public class ThriftValidation public static void validateRange(CFMetaData metadata, ColumnParent column_parent, SliceRange range) throws org.apache.cassandra.exceptions.InvalidRequestException { +if (range.count < 0) +throw new org.apache.cassandra.exceptions.InvalidRequestException("get_slice requires non-negative count"); + +if (range.start.remaining() > IColumn.MAX_NAME_LENGTH) +throw new org.apache.cassandra.exceptions.InvalidRequestException("range start length cannot be larger than " + IColumn.MAX_NAME_LENGTH); + +if (range.finish.remaining() > IColumn.MAX_NAME_LENGTH) +throw new org.apache.cassandra.exceptions.InvalidRequestException("range finish length cannot be larger than " + IColumn.MAX_NAME_LENGTH); + AbstractType comparator = metadata.getComparatorFor(column_parent.super_column); try { @@ -239,9 +248,6 @@ public class ThriftValidation throw new org.apache.cassandra.exceptions.InvalidRequestException(e.getMessage()); } -if (range.count < 0) -throw new org.apache.cassandra.exceptions.InvalidRequestException("get_slice requires non-negative count"); - Comparator orderedComparator = range.isReversed() ? comparator.reverseComparator : comparator; if (range.start.remaining() > 0 && range.finish.remaining() > 0
[jira] [Updated] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
[ https://issues.apache.org/jira/browse/CASSANDRA-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6521: - Attachment: 6521.txt Attaching Ben's patch with several modifications. > Thrift should validate SliceRange start and finish lengths > -- > > Key: CASSANDRA-6521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 > Project: Cassandra > Issue Type: Bug >Reporter: Ben Bromhead >Assignee: Ben Bromhead > Fix For: 1.2.14, 2.0.4 > > Attachments: 6521.txt > > > To quote [~benbromhead]: > bq. It appears that Cassandra does not check the length of a column name that > is part of a range predicate for a *_slice query before it serialises the > slice query to pass to the replicas. Names with a length greater than 0x > cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. > This further causes subsequent reads on the node to fail until Cassandra is > restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
[ https://issues.apache.org/jira/browse/CASSANDRA-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6521: - Assignee: Ben Bromhead (was: Aleksey Yeschenko) > Thrift should validate SliceRange start and finish lengths > -- > > Key: CASSANDRA-6521 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 > Project: Cassandra > Issue Type: Bug >Reporter: Ben Bromhead >Assignee: Ben Bromhead > Fix For: 1.2.14, 2.0.4 > > > To quote [~benbromhead]: > bq. It appears that Cassandra does not check the length of a column name that > is part of a range predicate for a *_slice query before it serialises the > slice query to pass to the replicas. Names with a length greater than 0x > cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. > This further causes subsequent reads on the node to fail until Cassandra is > restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6521) Thrift should validate SliceRange start and finish lengths
Aleksey Yeschenko created CASSANDRA-6521: Summary: Thrift should validate SliceRange start and finish lengths Key: CASSANDRA-6521 URL: https://issues.apache.org/jira/browse/CASSANDRA-6521 Project: Cassandra Issue Type: Bug Reporter: Ben Bromhead Assignee: Aleksey Yeschenko Fix For: 1.2.14, 2.0.4 To quote [~benbromhead]: bq. It appears that Cassandra does not check the length of a column name that is part of a range predicate for a *_slice query before it serialises the slice query to pass to the replicas. Names with a length greater than 0x cause an assertion error to occur in ByteBufferUtil.writeWithShortLength. This further causes subsequent reads on the node to fail until Cassandra is restarted -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5159) Expose droppable tombstone ratio stats over JMX
[ https://issues.apache.org/jira/browse/CASSANDRA-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855198#comment-13855198 ] Marcus Eriksson commented on CASSANDRA-5159: did you wait gc_grace time after doing the deletes? tombstones are not droppable until after that time. > Expose droppable tombstone ratio stats over JMX > --- > > Key: CASSANDRA-5159 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5159 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.2.0 >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 1.2.1 > > Attachments: > 0001-Expose-the-ratio-of-droppable-tombstones-over-JMX.patch > > > Could be nice to be able to jmx-monitor the tombstone histogram collected > during compactions to see how many gc-able tombstones there are left due to > the row existing in other sstables -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-4206) AssertionError: originally calculated column size of 629444349 but now it is 588008950
[ https://issues.apache.org/jira/browse/CASSANDRA-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855152#comment-13855152 ] Christian Spriegel commented on CASSANDRA-4206: --- I can confirm that this error also occurs with multithreaded_compaction=false. > AssertionError: originally calculated column size of 629444349 but now it is > 588008950 > -- > > Key: CASSANDRA-4206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4206 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.0.9 > Environment: Debian Squeeze Linux, kernel 2.6.32, sun-java6-bin > 6.26-0squeeze1 >Reporter: Patrik Modesto > > I've 4 node cluster of Cassandra 1.0.9. There is a rfTest3 keyspace with RF=3 > and one CF with two secondary indexes. I'm importing data into this CF using > Hadoop Mapreduce job, each row has less than 10 colkumns. From JMX: > MaxRowSize: 1597 > MeanRowSize: 369 > And there are some tens of millions of rows. > It's write-heavy usage and there is a big pressure on each node, there are > quite some dropped mutations on each node. After ~12 hours of inserting I see > these assertion exceptiona on 3 out of four nodes: > {noformat} > ERROR 06:25:40,124 Fatal exception in thread Thread[HintedHandoff:1,1,main] > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.AssertionError: originally calculated column size of 629444349 but > now it is 588008950 >at > org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:388) >at > org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:256) >at > org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:84) >at > org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:437) >at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) >at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >at java.lang.Thread.run(Thread.java:662) > Caused by: java.util.concurrent.ExecutionException: > java.lang.AssertionError: originally calculated column size of > 629444349 but now it is 588008950 >at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) >at java.util.concurrent.FutureTask.get(FutureTask.java:83) >at > org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:384) >... 7 more > Caused by: java.lang.AssertionError: originally calculated column size > of 629444349 but now it is 588008950 >at > org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124) >at > org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160) >at > org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:161) >at > org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:380) >at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >at java.util.concurrent.FutureTask.run(FutureTask.java:138) >... 3 more > {noformat} > Few lines regarding Hints from the output.log: > {noformat} > INFO 06:21:26,202 Compacting large row > system/HintsColumnFamily:7000 (1712834057 bytes) > incrementally > INFO 06:22:52,610 Compacting large row > system/HintsColumnFamily:1000 (2616073981 bytes) > incrementally > INFO 06:22:59,111 flushing high-traffic column family CFS(Keyspace='system', > ColumnFamily='HintsColumnFamily') (estimated 305147360 bytes) > INFO 06:22:59,813 Enqueuing flush of > Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes, > 7452 ops) > INFO 06:22:59,814 Writing > Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes, > 7452 ops) > {noformat} > I think the problem may be somehow connected to an IntegerType secondary > index. I had a different problem with CF with two secondary indexes, the > first UTF8Type, the second IntegerType. After a few hours of inserting data > in the afternoon and midnight repair+compact, the next day I couldn't find > any row using the IntegerType secondary index. The output was like this: > {noformat} > [default@rfTest3] get IndexTest where col1 = > '3230727:http://zaskolak.cz/download.php'; > --- > RowKey: 3230727:8383582:http://zaskolak.cz/download.php > => (column=col1, value=3230727:http://zaskolak.cz/download.php, > tim