[jira] [Commented] (CASSANDRA-7871) Reduce compaction IO in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125284#comment-14125284 ] Marcus Eriksson commented on CASSANDRA-7871: Have not had time to properly verify/confirm your calculations yet, but I have a few first comments/thoughts; * https://issues.apache.org/jira/browse/CASSANDRA-5727 has some background on why we picked 160MB as a default sstable size (yes, we should redo the same benchmarks with different level multipliers) * With level multiplier=4 the data size calculation is a bit off - it would contain less than 75% of the data in the top level (but maybe still good enough?) * Having more levels means hitting more sstables when doing reads and, I guess, that this is why 10 was chosen in the first place, as a tradeoff between compaction io and number of sstables to touch during reads I'll get back with a more thorough review soon > Reduce compaction IO in LCS > > > Key: CASSANDRA-7871 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7871 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Dan Hendry >Assignee: Dan Hendry > Fix For: 2.0.11 > > Attachments: LeveledCompactionImprovement-2.0.10.patch, > experiment.png, levelmultiplier.png, sstablesize.png > > > I have found LCS to be superior to STCS in almost every way - except for the > fact that it requires significantly more IO (a well advertised property). In > leveled compaction, L ~n+1~ is 10 times larger than L ~n~ so generally 1+10 > sstables need to be compacted to promote one sstable into the next level. For > certain workloads, this practically this means only 1/(10+1)=9% of the IO, > specifically write IO, is doing ‘useful’ work. > But why is each level 10 times larger? Why 10? Its a pretty looking number > and all but thats not a very good reason to choose it. If we chose 5 or even > 2 we could reduce the ‘wasted’ io required to promote an sstable to the next > level - of course at the expense of requiring more levels. I have not been > able to find justification for this choice in either cassandra or leveldb > itself. I would like to introduce a new parameter, the leveling multiplier, > which controls the desired size difference between L ~n~ and L ~n+1~. > First and foremost, a little math. Lets assume we have a CF of a fixed size > that is receiving continuous new data (ie: data is expiring due to TTLs or is > being overwritten). I believe the number of levels required is approximately > (see note 1): > {noformat}data size = (sstable size)*(leveling multiplier)^(level > count){noformat} > Which, when solving for the level count, becomes: > {noformat}level count = log((data size)/(sstable size))/log(leveling > multiplier){noformat} > The amount of compaction write IO required over the lifetime of a particular > piece of data (excluding compactions in L0) is: > {noformat}write IO = (flush IO) + (promotion IO)*(level count) > write IO = 1 + (1 + (level multiplier))*log((data size)/(sstable > size))/log(leveling multiplier){noformat} > So ultimately, the the relationship between write IO and the level multiplier > is f\(x) = (1 + x)/log\(x) which is optimal at 3.59, or 4 if we round to the > nearest integer. Also note that write IO is proportional to log((data > size)/(sstable size)) which suggests using larger sstables would also reduce > disk IO. > As one final analytical step we can add the following term to approximate STC > in L0 (which is not actually how its implemented but should be close enough > for moderate sstable sizes): > {noformat}L0 write IO = max(0, floor(log((sstable size)/(flush > size))/log(4))){noformat} > The following two graphs illustrate the predicted compaction requirements as > a function of the leveling multiplier and sstable size: > !levelmultiplier.png!!sstablesize.png! > In terms of empirically verifying the expected results, I set up three > cassandra nodes, node A having a leveling multiplier of 10 and sstable size > if 160 MB (current cassandra defaults), node B with multiplier 4 and size 160 > MB, and node C with multiplier 4 and size 1024 MB. I used a simple write only > workload which inserted data having a TTL of 2 days at 1 MB/second (see note > 2). Compaction throttling was disabled and gc_grace was 60 seconds. All nodes > had dedicated data disks and IO measurements were for the data disks only. > !experiment.png! > ||Measure||Node A (10, 160MB)||Node B (4, 160MB)||Node C (4, 1024MB)|| > |Predicted IO Rate|34.4 MB/s|26.2 MB/s|20.5 MB/s| > |Predicted Improvement|n/a|23.8%|40.4%| > |Predicted Number of Levels (Expected Dataset of 169 GB)|3.0|5.0|3.7| > |Experimental IO Rate|32.0 MB/s|28.0 MB/s|20.4 MB/s| > |Experimental Improvement|n/a|12.4%|*36.3%*|
[jira] [Comment Edited] (CASSANDRA-7871) Reduce compaction IO in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125284#comment-14125284 ] Marcus Eriksson edited comment on CASSANDRA-7871 at 9/8/14 7:50 AM: Have not had time to properly verify your calculations yet, but I have a few first comments/thoughts; * https://issues.apache.org/jira/browse/CASSANDRA-5727 has some background on why we picked 160MB as a default sstable size (yes, we should redo the same benchmarks with different level multipliers) * With level multiplier=4 the data size calculation is a bit off - it would contain less than 75% of the data in the top level (but maybe still good enough?) * Having more levels means hitting more sstables when doing reads and, I guess, that this is why 10 was chosen in the first place, as a tradeoff between compaction io and number of sstables to touch during reads I'll get back with a more thorough review soon was (Author: krummas): Have not had time to properly verify/confirm your calculations yet, but I have a few first comments/thoughts; * https://issues.apache.org/jira/browse/CASSANDRA-5727 has some background on why we picked 160MB as a default sstable size (yes, we should redo the same benchmarks with different level multipliers) * With level multiplier=4 the data size calculation is a bit off - it would contain less than 75% of the data in the top level (but maybe still good enough?) * Having more levels means hitting more sstables when doing reads and, I guess, that this is why 10 was chosen in the first place, as a tradeoff between compaction io and number of sstables to touch during reads I'll get back with a more thorough review soon > Reduce compaction IO in LCS > > > Key: CASSANDRA-7871 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7871 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Dan Hendry >Assignee: Dan Hendry > Fix For: 2.0.11 > > Attachments: LeveledCompactionImprovement-2.0.10.patch, > experiment.png, levelmultiplier.png, sstablesize.png > > > I have found LCS to be superior to STCS in almost every way - except for the > fact that it requires significantly more IO (a well advertised property). In > leveled compaction, L ~n+1~ is 10 times larger than L ~n~ so generally 1+10 > sstables need to be compacted to promote one sstable into the next level. For > certain workloads, this practically this means only 1/(10+1)=9% of the IO, > specifically write IO, is doing ‘useful’ work. > But why is each level 10 times larger? Why 10? Its a pretty looking number > and all but thats not a very good reason to choose it. If we chose 5 or even > 2 we could reduce the ‘wasted’ io required to promote an sstable to the next > level - of course at the expense of requiring more levels. I have not been > able to find justification for this choice in either cassandra or leveldb > itself. I would like to introduce a new parameter, the leveling multiplier, > which controls the desired size difference between L ~n~ and L ~n+1~. > First and foremost, a little math. Lets assume we have a CF of a fixed size > that is receiving continuous new data (ie: data is expiring due to TTLs or is > being overwritten). I believe the number of levels required is approximately > (see note 1): > {noformat}data size = (sstable size)*(leveling multiplier)^(level > count){noformat} > Which, when solving for the level count, becomes: > {noformat}level count = log((data size)/(sstable size))/log(leveling > multiplier){noformat} > The amount of compaction write IO required over the lifetime of a particular > piece of data (excluding compactions in L0) is: > {noformat}write IO = (flush IO) + (promotion IO)*(level count) > write IO = 1 + (1 + (level multiplier))*log((data size)/(sstable > size))/log(leveling multiplier){noformat} > So ultimately, the the relationship between write IO and the level multiplier > is f\(x) = (1 + x)/log\(x) which is optimal at 3.59, or 4 if we round to the > nearest integer. Also note that write IO is proportional to log((data > size)/(sstable size)) which suggests using larger sstables would also reduce > disk IO. > As one final analytical step we can add the following term to approximate STC > in L0 (which is not actually how its implemented but should be close enough > for moderate sstable sizes): > {noformat}L0 write IO = max(0, floor(log((sstable size)/(flush > size))/log(4))){noformat} > The following two graphs illustrate the predicted compaction requirements as > a function of the leveling multiplier and sstable size: > !levelmultiplier.png!!sstablesize.png! > In terms of empirically verifying the expected results, I set up three > cassandra nodes, node A having a level
[jira] [Commented] (CASSANDRA-7526) Defining UDFs using scripting language directly from CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125309#comment-14125309 ] Benjamin Lerer commented on CASSANDRA-7526: --- I just noticed that a language like Groovy does not seems to use by default InvokeDynamic (http://groovy.codehaus.org/InvokeDynamic+support).I guess that the same problem might exist with other languages. Could you make sure that the scripting language are always compiled to make use of InvokeDynamic? > Defining UDFs using scripting language directly from CQL > > > Key: CASSANDRA-7526 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7526 > Project: Cassandra > Issue Type: New Feature >Reporter: Sylvain Lebresne >Assignee: Robert Stupp > Fix For: 3.0 > > Attachments: 7526.txt, 7526v2.txt, 7526v3.txt, 7526v4.txt, 7526v5.txt > > > In CASSANDRA-7395 we'll introduce the ability to define user functions by > dropping a java class server side. While this is a good first step and a good > option to have in any case, it would be nice to provide a simpler way to > define those functions directly from CQL. And while we probably don't want to > re-invent a new programming language inside CQL, we can reuse one. Typically, > with java 8, we could use nashorn. This would allow a syntax along the lines > of: > {noformat} > CREATE FUNCTION sum (a bigint, b bigint) bigint AS { return a + b; } > {noformat} > Note that in this, everything before the AS will be parsed by us, which we'll > probably want because we'll probably need to have the types of > arguments/return in practice anyway, and it's a good idea to reuse CQL types. > The expression after the AS will be given to Nashorn however. > Please note that in theory we could ultimately support multiple language > after the AS. However, I'd like to focus on supporting just one for this > ticket and I'm keen on using javascript through Nashorn because as it's the > one that will ship with java from now on, it feels like a safe default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7874) Validate functionality of different JSR-223 providers in UDFs
[ https://issues.apache.org/jira/browse/CASSANDRA-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125363#comment-14125363 ] Robert Stupp commented on CASSANDRA-7874: - {{invokedynamic}} notes: Currently only Groovy and JRuby (on Java8) support invokedynamic. * Groovy provides two variants - one with and one without InvokeDynamic - the readme in contains instructions to use the one with InvokeDynamic. * Jython currently targets Java6 only. They want to switch to J7 + invokedynamic in Jython 3. * Scala 2.10 has J6 support only. 2.11 has experimental indy support. 2.12 introduces an upgrade directly to Java8 (https://stackoverflow.com/questions/14285894/advantages-of-scala-emitting-bytecode-for-the-jvm-1-7). * JRuby uses indy by default on J8 but not on J7 according to https://github.com/jruby/jruby/wiki/PerformanceTuning. > Validate functionality of different JSR-223 providers in UDFs > - > > Key: CASSANDRA-7874 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7874 > Project: Cassandra > Issue Type: Task > Components: Core >Reporter: Robert Stupp >Assignee: Robert Stupp > Labels: udf > Attachments: 7874.txt > > > CASSANDRA-7526 introduces ability to support optional JSR-223 providers like > Clojure, Jython, Groovy or JRuby. > This ticket is about to test functionality with these providers but not to > include them in C* distribution. > Expected result is a "how to" document, wiki page or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7526) Defining UDFs using scripting language directly from CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125364#comment-14125364 ] Robert Stupp commented on CASSANDRA-7526: - [~blerer] Answer in [this comment|https://issues.apache.org/jira/browse/CASSANDRA-7874?focusedCommentId=14125363&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14125363] of CASSANDRA-7874 > Defining UDFs using scripting language directly from CQL > > > Key: CASSANDRA-7526 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7526 > Project: Cassandra > Issue Type: New Feature >Reporter: Sylvain Lebresne >Assignee: Robert Stupp > Fix For: 3.0 > > Attachments: 7526.txt, 7526v2.txt, 7526v3.txt, 7526v4.txt, 7526v5.txt > > > In CASSANDRA-7395 we'll introduce the ability to define user functions by > dropping a java class server side. While this is a good first step and a good > option to have in any case, it would be nice to provide a simpler way to > define those functions directly from CQL. And while we probably don't want to > re-invent a new programming language inside CQL, we can reuse one. Typically, > with java 8, we could use nashorn. This would allow a syntax along the lines > of: > {noformat} > CREATE FUNCTION sum (a bigint, b bigint) bigint AS { return a + b; } > {noformat} > Note that in this, everything before the AS will be parsed by us, which we'll > probably want because we'll probably need to have the types of > arguments/return in practice anyway, and it's a good idea to reuse CQL types. > The expression after the AS will be given to Nashorn however. > Please note that in theory we could ultimately support multiple language > after the AS. However, I'd like to focus on supporting just one for this > ticket and I'm keen on using javascript through Nashorn because as it's the > one that will ship with java from now on, it feels like a safe default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-7898) IndexOutOfBoundsException comparing incompatible reversed types in DynamicCompositeType
Tim Whittington created CASSANDRA-7898: -- Summary: IndexOutOfBoundsException comparing incompatible reversed types in DynamicCompositeType Key: CASSANDRA-7898 URL: https://issues.apache.org/jira/browse/CASSANDRA-7898 Project: Cassandra Issue Type: Bug Components: Core Reporter: Tim Whittington In the following setup, an IndexOutOfBoundsException is often observed: * A DynamicCompositeType value is used in a column name (e.g. in a CQL3 cluster key) * Two incompatible types (e.g. Int32Type and DoubleType) are used in the dynamic composite value in their reversed form (e.g. ReversedType(Int32Type) and ReversedType(DoubleType) * Values for the incompatible types are inserted * One of various scenarios occurs that trigger comparison of the column names The exception can be variously observed (sometimes not immediately) during query execution, memtable flushing, commit log replay etc. In some cases this can prevent Cassandra server startup (e.g. during commit log replay). Typical stack traces follow: {noformat} java.lang.RuntimeException: java.lang.IndexOutOfBoundsException at org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:1968) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Buffer.java:538) at java.nio.HeapByteBuffer.getDouble(HeapByteBuffer.java:512) at org.apache.cassandra.utils.ByteBufferUtil.toDouble(ByteBufferUtil.java:431) at org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:33) at org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:25) at org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:142) at org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:45) at org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:28) at org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:73) at org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:30) at org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) at org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) at edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538) at edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108) at edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) at edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) at edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) at edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059) at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023) at edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985) at org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:319) at org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:191) at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226) at org.apache.cassandra.db.Memtable.put(Memtable.java:173) at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:900) at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:374) at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211) at org.apache.cassandra.service.StorageProxy$7.runMayThrow(StorageProxy.java:952) at org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:1964) {noformat} {noformat} java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Buffer.java:538) at java.nio.HeapByteBuffer.getDouble(HeapByteBuffer.java:512) at org.apache.cassandra.utils.ByteBufferUtil.toDouble(ByteBufferUtil.java:431) at org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:33
[jira] [Updated] (CASSANDRA-7898) IndexOutOfBoundsException comparing incompatible reversed types in DynamicCompositeType
[ https://issues.apache.org/jira/browse/CASSANDRA-7898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Whittington updated CASSANDRA-7898: --- Attachment: cassandra-2.0-7898.txt The root cause of this is that comparator selection in DynamicComposite inspects only the Java class identity of the AbstractType of the two parts of the comparison, and if they are the same uses the first AbstractType instance as the comparator. If the top level type of both values is ReversedType, then the first such type is used as a comparator to compose and compare both values, but the other value may be incompatible (ReversedType not being a singleton type) - any such comparison where the comparator expects more bytes than are available in the second value results in an IndexOutOfBoundsException. Attached patch + unit test that fixes this. > IndexOutOfBoundsException comparing incompatible reversed types in > DynamicCompositeType > --- > > Key: CASSANDRA-7898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7898 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Tim Whittington > Attachments: cassandra-2.0-7898.txt > > > In the following setup, an IndexOutOfBoundsException is often observed: > * A DynamicCompositeType value is used in a column name (e.g. in a CQL3 > cluster key) > * Two incompatible types (e.g. Int32Type and DoubleType) are used in the > dynamic composite value in their reversed form (e.g. ReversedType(Int32Type) > and ReversedType(DoubleType) > * Values for the incompatible types are inserted > * One of various scenarios occurs that trigger comparison of the column names > The exception can be variously observed (sometimes not immediately) during > query execution, memtable flushing, commit log replay etc. In some cases this > can prevent Cassandra server startup (e.g. during commit log replay). > Typical stack traces follow: > {noformat} > java.lang.RuntimeException: java.lang.IndexOutOfBoundsException > at > org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:1968) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.IndexOutOfBoundsException > at java.nio.Buffer.checkIndex(Buffer.java:538) > at java.nio.HeapByteBuffer.getDouble(HeapByteBuffer.java:512) > at > org.apache.cassandra.utils.ByteBufferUtil.toDouble(ByteBufferUtil.java:431) > at > org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:33) > at > org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:25) > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:142) > at > org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:45) > at > org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:28) > at > org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:73) > at > org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:30) > at > org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at > org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at > edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059) > at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023) > at > edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985) > at > org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:319) > at > org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDe
[jira] [Commented] (CASSANDRA-7360) CQLSSTableWriter consumes all memory for table with compound primary key
[ https://issues.apache.org/jira/browse/CASSANDRA-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125416#comment-14125416 ] Benjamin Lerer commented on CASSANDRA-7360: --- Just one nitpick: you do not need to create an ioe field as Throwable allow exception nesting. You can just call the RuntimeException(Throwable cause) constructor via super(ioe) and retrieve the Exception via getCause() in rawAddRow. > CQLSSTableWriter consumes all memory for table with compound primary key > > > Key: CASSANDRA-7360 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7360 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Xu Zhongxing >Assignee: Sylvain Lebresne > Fix For: 2.0.11 > > Attachments: 7360-v2.txt, 7360.txt > > > When using CQLSSTableWriter to write a table with compound primary key, if > the partition key is identical for a huge amount of records, the sync() > method is never called, and the memory usage keeps growing until the memory > is exhausted. > Could the code be improved to do sync() even when there is no new row > created? The relevant code is in SSTableSimpleUnsortedWriter.java and > AbstractSSTableSimpleWriter.java. I am new to the code and cannot produce a > reasonable patch for now. > The problem can be reproduced by the following test case: > {code} > import org.apache.cassandra.io.sstable.CQLSSTableWriter; > import org.apache.cassandra.exceptions.InvalidRequestException; > import java.io.IOException; > import java.util.UUID; > class SS { > public static void main(String[] args) { > String schema = "create table test.t (x uuid, y uuid, primary key (x, > y))"; > String insert = "insert into test.t (x, y) values (?, ?)"; > CQLSSTableWriter writer = CQLSSTableWriter.builder() > .inDirectory("/tmp/test/t") > .forTable(schema).withBufferSizeInMB(32) > .using(insert).build(); > UUID id = UUID.randomUUID(); > try { > for (int i = 0; i < 5000; i++) { > UUID id2 = UUID.randomUUID(); > writer.addRow(id, id2); > } > writer.close(); > } catch (Exception e) { > System.err.println("hell"); > } > } > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-7899) SSL does not work in cassandra-cli
Zdenek Ott created CASSANDRA-7899: - Summary: SSL does not work in cassandra-cli Key: CASSANDRA-7899 URL: https://issues.apache.org/jira/browse/CASSANDRA-7899 Project: Cassandra Issue Type: Bug Components: Tools Environment: Linux 2.6.32-431.20.3.el6.x86_64 #1 SMP Thu Jun 19 21:14:45 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) [cqlsh 4.1.1 | Cassandra 2.0.10 | CQL spec 3.1.1 | Thrift protocol 19.39.0] Reporter: Zdenek Ott When specify transport factory parameter '-tf org.apache.cassandra.cli.transport.SSLTransportFactory' it throws exception, see below, because SSLTransportFactory extends TTransportFactory not ITransportFactory. Exception in thread "main" java.lang.IllegalArgumentException: Cannot create a transport factory 'org.apache.cassandra.cli.transport.SSLTransportFactory'. at org.apache.cassandra.cli.CliOptions.validateAndSetTransportFactory(CliOptions.java:288) at org.apache.cassandra.cli.CliOptions.processArgs(CliOptions.java:223) at org.apache.cassandra.cli.CliMain.main(CliMain.java:230) Caused by: java.lang.IllegalArgumentException: transport factory 'org.apache.cassandra.cli.transport.SSLTransportFactory' not derived from ITransportFactory at org.apache.cassandra.cli.CliOptions.validateAndSetTransportFactory(CliOptions.java:282) ... 2 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-7900) Snapshot tests are not platform neutral
Philip Thompson created CASSANDRA-7900: -- Summary: Snapshot tests are not platform neutral Key: CASSANDRA-7900 URL: https://issues.apache.org/jira/browse/CASSANDRA-7900 Project: Cassandra Issue Type: Test Environment: Windows Reporter: Philip Thompson Assignee: Philip Thompson The various snapshot and commit log archiving tests in snapshot_test.py are failing on Windows platforms. This appears to be the fault of test behavior due to extensive operations in the file system that fail to consider the test may be running on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7900) Snapshot tests are not platform neutral
[ https://issues.apache.org/jira/browse/CASSANDRA-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125520#comment-14125520 ] Joshua McKenzie commented on CASSANDRA-7900: What branch are you using? I'd expect this behavior on 2.1 but these kinds of problems should be resolved on trunk after CASSANDRA-4050. > Snapshot tests are not platform neutral > --- > > Key: CASSANDRA-7900 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7900 > Project: Cassandra > Issue Type: Test > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson > Labels: Windows > > The various snapshot and commit log archiving tests in snapshot_test.py are > failing on Windows platforms. This appears to be the fault of test behavior > due to extensive operations in the file system that fail to consider the test > may be running on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7899) SSL does not work in cassandra-cli
[ https://issues.apache.org/jira/browse/CASSANDRA-7899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zdenek Ott updated CASSANDRA-7899: -- Environment: Linux 2.6.32-431.20.3.el6.x86_64 #1 SMP Thu Jun 19 21:14:45 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux java version "1.7.0_17" Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) [cqlsh 4.1.1 | Cassandra 2.0.10 | CQL spec 3.1.1 | Thrift protocol 19.39.0] was: Linux 2.6.32-431.20.3.el6.x86_64 #1 SMP Thu Jun 19 21:14:45 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) [cqlsh 4.1.1 | Cassandra 2.0.10 | CQL spec 3.1.1 | Thrift protocol 19.39.0] > SSL does not work in cassandra-cli > -- > > Key: CASSANDRA-7899 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7899 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Linux 2.6.32-431.20.3.el6.x86_64 #1 SMP Thu Jun 19 > 21:14:45 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux > java version "1.7.0_17" > Java(TM) SE Runtime Environment (build 1.7.0_17-b02) > Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) > [cqlsh 4.1.1 | Cassandra 2.0.10 | CQL spec 3.1.1 | Thrift protocol 19.39.0] >Reporter: Zdenek Ott > > When specify transport factory parameter '-tf > org.apache.cassandra.cli.transport.SSLTransportFactory' it throws exception, > see below, because SSLTransportFactory extends TTransportFactory not > ITransportFactory. > Exception in thread "main" java.lang.IllegalArgumentException: Cannot create > a transport factory 'org.apache.cassandra.cli.transport.SSLTransportFactory'. > at > org.apache.cassandra.cli.CliOptions.validateAndSetTransportFactory(CliOptions.java:288) > at > org.apache.cassandra.cli.CliOptions.processArgs(CliOptions.java:223) > at org.apache.cassandra.cli.CliMain.main(CliMain.java:230) > Caused by: java.lang.IllegalArgumentException: transport factory > 'org.apache.cassandra.cli.transport.SSLTransportFactory' not derived from > ITransportFactory > at > org.apache.cassandra.cli.CliOptions.validateAndSetTransportFactory(CliOptions.java:282) > ... 2 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7900) Snapshot tests are not platform neutral
[ https://issues.apache.org/jira/browse/CASSANDRA-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125652#comment-14125652 ] Joshua McKenzie commented on CASSANDRA-7900: We're still using a RandomAccessFile for our CommitLogSegments on trunk so I could see some tests causing headache if we're trying to remove those files while there's still handles to those files open. I don't expect this will be a problem with regular DB operations as we don't symlink or hard link to our CLS (that I know of off the top of my head) - the tests may need to be re-factored to work around any current timing issues related to CLS inter-test however. As for SSTR and snapshots - those should behave on trunk. > Snapshot tests are not platform neutral > --- > > Key: CASSANDRA-7900 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7900 > Project: Cassandra > Issue Type: Test > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson > Labels: Windows > > The various snapshot and commit log archiving tests in snapshot_test.py are > failing on Windows platforms. This appears to be the fault of test behavior > due to extensive operations in the file system that fail to consider the test > may be running on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7463) Update CQLSSTableWriter to allow parallel writing of SSTables on the same table within the same JVM
[ https://issues.apache.org/jira/browse/CASSANDRA-7463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-7463: -- Assignee: Carl Yeksigian > Update CQLSSTableWriter to allow parallel writing of SSTables on the same > table within the same JVM > --- > > Key: CASSANDRA-7463 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7463 > Project: Cassandra > Issue Type: Improvement >Reporter: Johnny Miller >Assignee: Carl Yeksigian > > Currently it is not possible to programatically write multiple SSTables for > the same table in parallel using the CQLSSTableWriter. This is quite a > limitation and the workaround of attempting to do this in a separate JVM is > not a great solution. > See: > http://stackoverflow.com/questions/24396902/using-cqlsstablewriter-concurrently -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
Philip Thompson created CASSANDRA-7901: -- Summary: Implement -f functionality in stop-server.bat Key: CASSANDRA-7901 URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 Project: Cassandra Issue Type: Improvement Environment: Windows Reporter: Philip Thompson Assignee: Philip Thompson Priority: Minor Fix For: 2.1.1, 3.0 Stop-server.bat lists "-f" as an argument but does not handle it inside of stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7849) Server logged error messages (in binary protocol) for unexpected exceptions could be more helpful
[ https://issues.apache.org/jira/browse/CASSANDRA-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125698#comment-14125698 ] graham sanderson commented on CASSANDRA-7849: - # So the vote is to change all logging from this class to debug? # We should probably add something to NEWS.txt don't you think, about turning on debugging for the class to debug network issues or binary client driver problems #* Do I do that as part of the patch > Server logged error messages (in binary protocol) for unexpected exceptions > could be more helpful > - > > Key: CASSANDRA-7849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7849 > Project: Cassandra > Issue Type: Improvement >Reporter: graham sanderson > Fix For: 1.2.19, 2.0.11 > > Attachments: cassandra-1.2-7849.txt > > > From time to time (actually quite frequently) we get error messages in the > server logs like this > {code} > ERROR [Native-Transport-Requests:288] 2014-08-29 04:48:07,118 > ErrorMessage.java (line 222) Unexpected exception during request > java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) > at sun.nio.ch.IOUtil.read(IOUtil.java:192) > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) > at > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) > at > org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) > at > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > {code} > These particular cases are almost certainly problems with the client driver, > client machine, client process, however after the fact this particular > exception is practically impossible to debug because there is no indication > in the underlying JVM/netty exception of who the peer was. I should note we > have lots of different types of applications running against the cluster so > it is very hard to correlate these to anything -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7849) Server logged error messages (in binary protocol) for unexpected exceptions could be more helpful
[ https://issues.apache.org/jira/browse/CASSANDRA-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125698#comment-14125698 ] graham sanderson edited comment on CASSANDRA-7849 at 9/8/14 4:05 PM: - # So the vote is to change all logging from this class to debug? # We should probably add something to NEWS.txt don't you think, about turning on debugging for the class to debug network issues or binary client driver problems #* Do I do that as part of the patch? was (Author: graham sanderson): # So the vote is to change all logging from this class to debug? # We should probably add something to NEWS.txt don't you think, about turning on debugging for the class to debug network issues or binary client driver problems #* Do I do that as part of the patch > Server logged error messages (in binary protocol) for unexpected exceptions > could be more helpful > - > > Key: CASSANDRA-7849 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7849 > Project: Cassandra > Issue Type: Improvement >Reporter: graham sanderson > Fix For: 1.2.19, 2.0.11 > > Attachments: cassandra-1.2-7849.txt > > > From time to time (actually quite frequently) we get error messages in the > server logs like this > {code} > ERROR [Native-Transport-Requests:288] 2014-08-29 04:48:07,118 > ErrorMessage.java (line 222) Unexpected exception during request > java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) > at sun.nio.ch.IOUtil.read(IOUtil.java:192) > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) > at > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) > at > org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) > at > org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) > at > org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > {code} > These particular cases are almost certainly problems with the client driver, > client machine, client process, however after the fact this particular > exception is practically impossible to debug because there is no indication > in the underlying JVM/netty exception of who the peer was. I should note we > have lots of different types of applications running against the cluster so > it is very hard to correlate these to anything -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7443) SSTable Pluggability v2
[ https://issues.apache.org/jira/browse/CASSANDRA-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-7443: -- Reviewer: Benedict > SSTable Pluggability v2 > --- > > Key: CASSANDRA-7443 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7443 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 3.0 > > Attachments: 7443-refactor-v1.txt, 7443-testformat-v1.txt > > > As part of a wider effort to improve the performance of our storage engine we > will need to support basic pluggability of the SSTable reader/writer. We > primarily need this to support the current SSTable format and new SSTable > format in the same version. This will also let us encapsulate the changes in > a single layer vs forcing the whole engine to change at once. > We previously discussed how to accomplish this in CASSANDRA-3067 > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-7818) Improve compaction logging
[ https://issues.apache.org/jira/browse/CASSANDRA-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minh Do reassigned CASSANDRA-7818: -- Assignee: Minh Do > Improve compaction logging > -- > > Key: CASSANDRA-7818 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7818 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Minh Do >Priority: Minor > Labels: compaction, lhf > Fix For: 2.1.1 > > > We should log more information about compactions to be able to debug issues > more efficiently > * give each CompactionTask an id that we log (so that you can relate the > start-compaction-messages to the finished-compaction ones) > * log what level the sstables are taken from -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7828) New node cannot be joined if a value in composite type column is dropped (description updated)
[ https://issues.apache.org/jira/browse/CASSANDRA-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125848#comment-14125848 ] Aleksey Yeschenko commented on CASSANDRA-7828: -- +1 On merge w/ 2.1 should switch to argument-less Cell#isLive(). > New node cannot be joined if a value in composite type column is dropped > (description updated) > -- > > Key: CASSANDRA-7828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7828 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Igor Zubchenok >Assignee: Mikhail Stepura > Fix For: 2.0.11 > > Attachments: 1409959462180-myColumnFamily.zip, > CASSANDRA-2.0-7828.patch > > > I get a *RuntimeException* at new node system.log on bootstrapping a new DC: > {code:title=system.out - RuntimeException caused by IllegalArgumentException > in Buffer.limit|borderStyle=solid} > INFO [NonPeriodicTasks:1] 2014-08-26 15:43:01,030 SecondaryIndexManager.java > (line 137) Submitting index build of [myColumnFamily.myColumnFamily_myColumn] > for data in > SSTableReader(path='/var/lib/cassandra/data/testbug/myColumnFamily/testbug-myColumnFamily-jb-1-Data.db') > ERROR [CompactionExecutor:2] 2014-08-26 15:43:01,035 CassandraDaemon.java > (line 199) Exception in thread Thread[CompactionExecutor:2,1,main] > java.lang.IllegalArgumentException > at java.nio.Buffer.limit(Buffer.java:267) > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:587) > at > org.apache.cassandra.utils.ByteBufferUtil.readBytesWithShortLength(ByteBufferUtil.java:596) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:61) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at org.apache.cassandra.dht.LocalToken.compareTo(LocalToken.java:44) > at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:85) > at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:36) > at > java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:727) > at > java.util.concurrent.ConcurrentSkipListMap.findNode(ConcurrentSkipListMap.java:789) > at > java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:828) > at > java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1626) > at org.apache.cassandra.db.Memtable.resolve(Memtable.java:215) > at org.apache.cassandra.db.Memtable.put(Memtable.java:173) > at > org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:900) > at > org.apache.cassandra.db.index.AbstractSimplePerColumnSecondaryIndex.insert(AbstractSimplePerColumnSecondaryIndex.java:107) > at > org.apache.cassandra.db.index.SecondaryIndexManager.indexRow(SecondaryIndexManager.java:441) > at org.apache.cassandra.db.Keyspace.indexRow(Keyspace.java:413) > at > org.apache.cassandra.db.index.SecondaryIndexBuilder.build(SecondaryIndexBuilder.java:62) > at > org.apache.cassandra.db.compaction.CompactionManager$9.run(CompactionManager.java:834) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ERROR [NonPeriodicTasks:1] 2014-08-26 15:43:01,035 CassandraDaemon.java (line > 199) Exception in thread Thread[NonPeriodicTasks:1,5,main] > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.IllegalArgumentException > at > org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:413) > at > org.apache.cassandra.db.index.SecondaryIndexManager.maybeBuildSecondaryIndexes(SecondaryIndexManager.java:142) > at > org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:113) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
[jira] [Commented] (CASSANDRA-7828) New node cannot be joined if a value in composite type column is dropped (description updated)
[ https://issues.apache.org/jira/browse/CASSANDRA-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125943#comment-14125943 ] Mikhail Stepura commented on CASSANDRA-7828: [~iamaleksey] should we merge it into 2.1.0 as well? > New node cannot be joined if a value in composite type column is dropped > (description updated) > -- > > Key: CASSANDRA-7828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7828 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Igor Zubchenok >Assignee: Mikhail Stepura > Fix For: 2.0.11 > > Attachments: 1409959462180-myColumnFamily.zip, > CASSANDRA-2.0-7828.patch > > > I get a *RuntimeException* at new node system.log on bootstrapping a new DC: > {code:title=system.out - RuntimeException caused by IllegalArgumentException > in Buffer.limit|borderStyle=solid} > INFO [NonPeriodicTasks:1] 2014-08-26 15:43:01,030 SecondaryIndexManager.java > (line 137) Submitting index build of [myColumnFamily.myColumnFamily_myColumn] > for data in > SSTableReader(path='/var/lib/cassandra/data/testbug/myColumnFamily/testbug-myColumnFamily-jb-1-Data.db') > ERROR [CompactionExecutor:2] 2014-08-26 15:43:01,035 CassandraDaemon.java > (line 199) Exception in thread Thread[CompactionExecutor:2,1,main] > java.lang.IllegalArgumentException > at java.nio.Buffer.limit(Buffer.java:267) > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:587) > at > org.apache.cassandra.utils.ByteBufferUtil.readBytesWithShortLength(ByteBufferUtil.java:596) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:61) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at org.apache.cassandra.dht.LocalToken.compareTo(LocalToken.java:44) > at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:85) > at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:36) > at > java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:727) > at > java.util.concurrent.ConcurrentSkipListMap.findNode(ConcurrentSkipListMap.java:789) > at > java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:828) > at > java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1626) > at org.apache.cassandra.db.Memtable.resolve(Memtable.java:215) > at org.apache.cassandra.db.Memtable.put(Memtable.java:173) > at > org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:900) > at > org.apache.cassandra.db.index.AbstractSimplePerColumnSecondaryIndex.insert(AbstractSimplePerColumnSecondaryIndex.java:107) > at > org.apache.cassandra.db.index.SecondaryIndexManager.indexRow(SecondaryIndexManager.java:441) > at org.apache.cassandra.db.Keyspace.indexRow(Keyspace.java:413) > at > org.apache.cassandra.db.index.SecondaryIndexBuilder.build(SecondaryIndexBuilder.java:62) > at > org.apache.cassandra.db.compaction.CompactionManager$9.run(CompactionManager.java:834) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ERROR [NonPeriodicTasks:1] 2014-08-26 15:43:01,035 CassandraDaemon.java (line > 199) Exception in thread Thread[NonPeriodicTasks:1,5,main] > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.IllegalArgumentException > at > org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:413) > at > org.apache.cassandra.db.index.SecondaryIndexManager.maybeBuildSecondaryIndexes(SecondaryIndexManager.java:142) > at > org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:113) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >
[jira] [Commented] (CASSANDRA-7828) New node cannot be joined if a value in composite type column is dropped (description updated)
[ https://issues.apache.org/jira/browse/CASSANDRA-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125944#comment-14125944 ] Aleksey Yeschenko commented on CASSANDRA-7828: -- No, cassandra-2.1.0 branch will be removed soon, since 2.1.0 is done. > New node cannot be joined if a value in composite type column is dropped > (description updated) > -- > > Key: CASSANDRA-7828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7828 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Igor Zubchenok >Assignee: Mikhail Stepura > Fix For: 2.0.11 > > Attachments: 1409959462180-myColumnFamily.zip, > CASSANDRA-2.0-7828.patch > > > I get a *RuntimeException* at new node system.log on bootstrapping a new DC: > {code:title=system.out - RuntimeException caused by IllegalArgumentException > in Buffer.limit|borderStyle=solid} > INFO [NonPeriodicTasks:1] 2014-08-26 15:43:01,030 SecondaryIndexManager.java > (line 137) Submitting index build of [myColumnFamily.myColumnFamily_myColumn] > for data in > SSTableReader(path='/var/lib/cassandra/data/testbug/myColumnFamily/testbug-myColumnFamily-jb-1-Data.db') > ERROR [CompactionExecutor:2] 2014-08-26 15:43:01,035 CassandraDaemon.java > (line 199) Exception in thread Thread[CompactionExecutor:2,1,main] > java.lang.IllegalArgumentException > at java.nio.Buffer.limit(Buffer.java:267) > at > org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:587) > at > org.apache.cassandra.utils.ByteBufferUtil.readBytesWithShortLength(ByteBufferUtil.java:596) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:61) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at org.apache.cassandra.dht.LocalToken.compareTo(LocalToken.java:44) > at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:85) > at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:36) > at > java.util.concurrent.ConcurrentSkipListMap.findPredecessor(ConcurrentSkipListMap.java:727) > at > java.util.concurrent.ConcurrentSkipListMap.findNode(ConcurrentSkipListMap.java:789) > at > java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:828) > at > java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1626) > at org.apache.cassandra.db.Memtable.resolve(Memtable.java:215) > at org.apache.cassandra.db.Memtable.put(Memtable.java:173) > at > org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:900) > at > org.apache.cassandra.db.index.AbstractSimplePerColumnSecondaryIndex.insert(AbstractSimplePerColumnSecondaryIndex.java:107) > at > org.apache.cassandra.db.index.SecondaryIndexManager.indexRow(SecondaryIndexManager.java:441) > at org.apache.cassandra.db.Keyspace.indexRow(Keyspace.java:413) > at > org.apache.cassandra.db.index.SecondaryIndexBuilder.build(SecondaryIndexBuilder.java:62) > at > org.apache.cassandra.db.compaction.CompactionManager$9.run(CompactionManager.java:834) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ERROR [NonPeriodicTasks:1] 2014-08-26 15:43:01,035 CassandraDaemon.java (line > 199) Exception in thread Thread[NonPeriodicTasks:1,5,main] > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.IllegalArgumentException > at > org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:413) > at > org.apache.cassandra.db.index.SecondaryIndexManager.maybeBuildSecondaryIndexes(SecondaryIndexManager.java:142) > at > org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:113) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExec
[04/10] git commit: Don't index tombstones.
Don't index tombstones. patch by Mikhail Stepura; reviewed by Aleksey Yeschenko for CASSANDRA-7828 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6860d6fa Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6860d6fa Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6860d6fa Branch: refs/heads/trunk Commit: 6860d6fad9e6af90ff7c2f4359a0bdc1226135b5 Parents: 8bef260 Author: Mikhail Stepura Authored: Mon Sep 8 12:01:37 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:01:37 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6c601e5..435b124 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.2.19 + * Don't index tombstones (CASSANDRA-7828) * Don't allow compacted sstables to be marked as compacting (CASSANDRA-7145) * Track expired tombstones (CASSANDRA-7810) * Validate empty cell names from counter updates (CASSANDRA-7798) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java -- diff --git a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java index 7fefa13..8839228 100644 --- a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java +++ b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java @@ -443,7 +443,7 @@ public class SecondaryIndexManager { for (IColumn column : cf) { -if (index.indexes(column.name())) +if (column.isLive() && index.indexes(column.name())) ((PerColumnSecondaryIndex) index).insert(key, column); } }
[10/10] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/92881598 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/92881598 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/92881598 Branch: refs/heads/trunk Commit: 9288159898605f8f160b040dab46be2c089b30c8 Parents: d641053 ad9f51f Author: Mikhail Stepura Authored: Mon Sep 8 12:21:36 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:21:36 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/92881598/CHANGES.txt --
[02/10] git commit: Don't index tombstones.
Don't index tombstones. patch by Mikhail Stepura; reviewed by Aleksey Yeschenko for CASSANDRA-7828 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6860d6fa Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6860d6fa Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6860d6fa Branch: refs/heads/cassandra-2.0 Commit: 6860d6fad9e6af90ff7c2f4359a0bdc1226135b5 Parents: 8bef260 Author: Mikhail Stepura Authored: Mon Sep 8 12:01:37 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:01:37 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6c601e5..435b124 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.2.19 + * Don't index tombstones (CASSANDRA-7828) * Don't allow compacted sstables to be marked as compacting (CASSANDRA-7145) * Track expired tombstones (CASSANDRA-7810) * Validate empty cell names from counter updates (CASSANDRA-7798) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java -- diff --git a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java index 7fefa13..8839228 100644 --- a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java +++ b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java @@ -443,7 +443,7 @@ public class SecondaryIndexManager { for (IColumn column : cf) { -if (index.indexes(column.name())) +if (column.isLive() && index.indexes(column.name())) ((PerColumnSecondaryIndex) index).insert(key, column); } }
[01/10] git commit: Don't index tombstones.
Repository: cassandra Updated Branches: refs/heads/cassandra-1.2 8bef2609a -> 6860d6fad refs/heads/cassandra-2.0 e6f307f4e -> 8aa3c3a6a refs/heads/cassandra-2.1 f1f5f5fd3 -> ad9f51fc5 refs/heads/trunk d641053ab -> 928815989 Don't index tombstones. patch by Mikhail Stepura; reviewed by Aleksey Yeschenko for CASSANDRA-7828 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6860d6fa Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6860d6fa Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6860d6fa Branch: refs/heads/cassandra-1.2 Commit: 6860d6fad9e6af90ff7c2f4359a0bdc1226135b5 Parents: 8bef260 Author: Mikhail Stepura Authored: Mon Sep 8 12:01:37 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:01:37 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6c601e5..435b124 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.2.19 + * Don't index tombstones (CASSANDRA-7828) * Don't allow compacted sstables to be marked as compacting (CASSANDRA-7145) * Track expired tombstones (CASSANDRA-7810) * Validate empty cell names from counter updates (CASSANDRA-7798) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java -- diff --git a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java index 7fefa13..8839228 100644 --- a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java +++ b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java @@ -443,7 +443,7 @@ public class SecondaryIndexManager { for (IColumn column : cf) { -if (index.indexes(column.name())) +if (column.isLive() && index.indexes(column.name())) ((PerColumnSecondaryIndex) index).insert(key, column); } }
[07/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8aa3c3a6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8aa3c3a6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8aa3c3a6 Branch: refs/heads/trunk Commit: 8aa3c3a6a36d4fd308e5a7b518871ec72fa632fe Parents: e6f307f 6860d6f Author: Mikhail Stepura Authored: Mon Sep 8 12:06:09 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:06:09 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8aa3c3a6/CHANGES.txt -- diff --cc CHANGES.txt index 54fcc4f,435b124..747df0d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,86 -1,5 +1,87 @@@ -1.2.19 - * Don't index tombstones (CASSANDRA-7828) +2.0.11: + * Explicitly disallowing mixing multi-column and single-column + relations on clustering columns (CASSANDRA-7711) + * Better error message when condition is set on PK column (CASSANDRA-7804) + * Forbid re-adding dropped counter columns (CASSANDRA-7831) + * Fix CFMetaData#isThriftCompatible() for PK-only tables (CASSANDRA-7832) + * Always reject inequality on the partition key without token() + (CASSANDRA-7722) + * Always send Paxos commit to all replicas (CASSANDRA-7479) + * Make disruptor_thrift_server invocation pool configurable (CASSANDRA-7594) + * Make repair no-op when RF=1 (CASSANDRA-7864) + + +2.0.10 + * Don't send schema change responses and events for no-op DDL + statements (CASSANDRA-7600) + * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774) + * Configure system.paxos with LeveledCompactionStrategy (CASSANDRA-7753) + * Fix ALTER clustering column type from DateType to TimestampType when + using DESC clustering order (CASSANRDA-7797) + * Stop inheriting liveRatio and liveRatioComputedAt from previous + memtables (CASSANDRA-7796) + * Throw EOFException if we run out of chunks in compressed datafile + (CASSANDRA-7664) + * Throw InvalidRequestException when queries contain relations on entire + collection columns (CASSANDRA-7506) + * Fix PRSI handling of CQL3 row markers for row cleanup (CASSANDRA-7787) + * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577) + * Fix dropping collection when it's the last regular column (CASSANDRA-7744) + * Properly reject operations on list index with conditions (CASSANDRA-7499) + * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252) + * (cqlsh) Wait up to 10 sec for a tracing session (CASSANDRA-7222) + * Fix NPE in FileCacheService.sizeInBytes (CASSANDRA-7756) + * (cqlsh) cqlsh should automatically disable tracing when selecting + from system_traces (CASSANDRA-7641) + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752) + * Do not flush on truncate if durable_writes is false (CASSANDRA-7750) + * Give CRR a default input_cql Statement (CASSANDRA-7226) + * Better error message when adding a collection with the same name + than a previously dropped one (CASSANDRA-6276) + * Fix validation when adding static columns (CASSANDRA-7730) + * (Thrift) fix range deletion of supercolumns (CASSANDRA-7733) + * Fix potential AssertionError in RangeTombstoneList (CASSANDRA-7700) + * Validate arguments of blobAs* functions (CASSANDRA-7707) + * Fix potential AssertionError with 2ndary indexes (CASSANDRA-6612) + * Avoid logging CompactionInterrupted at ERROR (CASSANDRA-7694) + * Minor leak in sstable2jon (CASSANDRA-7709) + * Add cassandra.auto_bootstrap system property (CASSANDRA-7650) + * Remove CqlPagingRecordReader/CqlPagingInputFormat (CASSANDRA-7570) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * Update java driver (for hadoop) (CASSANDRA-7618) + * Fix truncate to always flush (CASSANDRA-7511) + * Remove shuffle and taketoken (CASSANDRA-7601) + * Switch liveRatio-related log messages to DEBUG (CASSANDRA-7467) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Always merge ranges owned by a single node (CASSANDRA-6930) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Fix ReversedType(DateType) mapping to native pr
[06/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8aa3c3a6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8aa3c3a6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8aa3c3a6 Branch: refs/heads/cassandra-2.0 Commit: 8aa3c3a6a36d4fd308e5a7b518871ec72fa632fe Parents: e6f307f 6860d6f Author: Mikhail Stepura Authored: Mon Sep 8 12:06:09 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:06:09 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8aa3c3a6/CHANGES.txt -- diff --cc CHANGES.txt index 54fcc4f,435b124..747df0d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,86 -1,5 +1,87 @@@ -1.2.19 - * Don't index tombstones (CASSANDRA-7828) +2.0.11: + * Explicitly disallowing mixing multi-column and single-column + relations on clustering columns (CASSANDRA-7711) + * Better error message when condition is set on PK column (CASSANDRA-7804) + * Forbid re-adding dropped counter columns (CASSANDRA-7831) + * Fix CFMetaData#isThriftCompatible() for PK-only tables (CASSANDRA-7832) + * Always reject inequality on the partition key without token() + (CASSANDRA-7722) + * Always send Paxos commit to all replicas (CASSANDRA-7479) + * Make disruptor_thrift_server invocation pool configurable (CASSANDRA-7594) + * Make repair no-op when RF=1 (CASSANDRA-7864) + + +2.0.10 + * Don't send schema change responses and events for no-op DDL + statements (CASSANDRA-7600) + * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774) + * Configure system.paxos with LeveledCompactionStrategy (CASSANDRA-7753) + * Fix ALTER clustering column type from DateType to TimestampType when + using DESC clustering order (CASSANRDA-7797) + * Stop inheriting liveRatio and liveRatioComputedAt from previous + memtables (CASSANDRA-7796) + * Throw EOFException if we run out of chunks in compressed datafile + (CASSANDRA-7664) + * Throw InvalidRequestException when queries contain relations on entire + collection columns (CASSANDRA-7506) + * Fix PRSI handling of CQL3 row markers for row cleanup (CASSANDRA-7787) + * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577) + * Fix dropping collection when it's the last regular column (CASSANDRA-7744) + * Properly reject operations on list index with conditions (CASSANDRA-7499) + * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252) + * (cqlsh) Wait up to 10 sec for a tracing session (CASSANDRA-7222) + * Fix NPE in FileCacheService.sizeInBytes (CASSANDRA-7756) + * (cqlsh) cqlsh should automatically disable tracing when selecting + from system_traces (CASSANDRA-7641) + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752) + * Do not flush on truncate if durable_writes is false (CASSANDRA-7750) + * Give CRR a default input_cql Statement (CASSANDRA-7226) + * Better error message when adding a collection with the same name + than a previously dropped one (CASSANDRA-6276) + * Fix validation when adding static columns (CASSANDRA-7730) + * (Thrift) fix range deletion of supercolumns (CASSANDRA-7733) + * Fix potential AssertionError in RangeTombstoneList (CASSANDRA-7700) + * Validate arguments of blobAs* functions (CASSANDRA-7707) + * Fix potential AssertionError with 2ndary indexes (CASSANDRA-6612) + * Avoid logging CompactionInterrupted at ERROR (CASSANDRA-7694) + * Minor leak in sstable2jon (CASSANDRA-7709) + * Add cassandra.auto_bootstrap system property (CASSANDRA-7650) + * Remove CqlPagingRecordReader/CqlPagingInputFormat (CASSANDRA-7570) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * Update java driver (for hadoop) (CASSANDRA-7618) + * Fix truncate to always flush (CASSANDRA-7511) + * Remove shuffle and taketoken (CASSANDRA-7601) + * Switch liveRatio-related log messages to DEBUG (CASSANDRA-7467) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Always merge ranges owned by a single node (CASSANDRA-6930) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Fix ReversedType(DateType) mapping to n
[05/10] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8aa3c3a6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8aa3c3a6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8aa3c3a6 Branch: refs/heads/cassandra-2.1 Commit: 8aa3c3a6a36d4fd308e5a7b518871ec72fa632fe Parents: e6f307f 6860d6f Author: Mikhail Stepura Authored: Mon Sep 8 12:06:09 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:06:09 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8aa3c3a6/CHANGES.txt -- diff --cc CHANGES.txt index 54fcc4f,435b124..747df0d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,86 -1,5 +1,87 @@@ -1.2.19 - * Don't index tombstones (CASSANDRA-7828) +2.0.11: + * Explicitly disallowing mixing multi-column and single-column + relations on clustering columns (CASSANDRA-7711) + * Better error message when condition is set on PK column (CASSANDRA-7804) + * Forbid re-adding dropped counter columns (CASSANDRA-7831) + * Fix CFMetaData#isThriftCompatible() for PK-only tables (CASSANDRA-7832) + * Always reject inequality on the partition key without token() + (CASSANDRA-7722) + * Always send Paxos commit to all replicas (CASSANDRA-7479) + * Make disruptor_thrift_server invocation pool configurable (CASSANDRA-7594) + * Make repair no-op when RF=1 (CASSANDRA-7864) + + +2.0.10 + * Don't send schema change responses and events for no-op DDL + statements (CASSANDRA-7600) + * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774) + * Configure system.paxos with LeveledCompactionStrategy (CASSANDRA-7753) + * Fix ALTER clustering column type from DateType to TimestampType when + using DESC clustering order (CASSANRDA-7797) + * Stop inheriting liveRatio and liveRatioComputedAt from previous + memtables (CASSANDRA-7796) + * Throw EOFException if we run out of chunks in compressed datafile + (CASSANDRA-7664) + * Throw InvalidRequestException when queries contain relations on entire + collection columns (CASSANDRA-7506) + * Fix PRSI handling of CQL3 row markers for row cleanup (CASSANDRA-7787) + * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577) + * Fix dropping collection when it's the last regular column (CASSANDRA-7744) + * Properly reject operations on list index with conditions (CASSANDRA-7499) + * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252) + * (cqlsh) Wait up to 10 sec for a tracing session (CASSANDRA-7222) + * Fix NPE in FileCacheService.sizeInBytes (CASSANDRA-7756) + * (cqlsh) cqlsh should automatically disable tracing when selecting + from system_traces (CASSANDRA-7641) + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752) + * Do not flush on truncate if durable_writes is false (CASSANDRA-7750) + * Give CRR a default input_cql Statement (CASSANDRA-7226) + * Better error message when adding a collection with the same name + than a previously dropped one (CASSANDRA-6276) + * Fix validation when adding static columns (CASSANDRA-7730) + * (Thrift) fix range deletion of supercolumns (CASSANDRA-7733) + * Fix potential AssertionError in RangeTombstoneList (CASSANDRA-7700) + * Validate arguments of blobAs* functions (CASSANDRA-7707) + * Fix potential AssertionError with 2ndary indexes (CASSANDRA-6612) + * Avoid logging CompactionInterrupted at ERROR (CASSANDRA-7694) + * Minor leak in sstable2jon (CASSANDRA-7709) + * Add cassandra.auto_bootstrap system property (CASSANDRA-7650) + * Remove CqlPagingRecordReader/CqlPagingInputFormat (CASSANDRA-7570) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * Update java driver (for hadoop) (CASSANDRA-7618) + * Fix truncate to always flush (CASSANDRA-7511) + * Remove shuffle and taketoken (CASSANDRA-7601) + * Switch liveRatio-related log messages to DEBUG (CASSANDRA-7467) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Always merge ranges owned by a single node (CASSANDRA-6930) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Fix ReversedType(DateType) mapping to n
[09/10] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0
Merge branch 'cassandra-2.0' into cassandra-2.1.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ad9f51fc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ad9f51fc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ad9f51fc Branch: refs/heads/trunk Commit: ad9f51fc5cc87ff744eee0ceb4abb05f5f3a3e24 Parents: f1f5f5f 8aa3c3a Author: Mikhail Stepura Authored: Mon Sep 8 12:20:51 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:20:51 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ad9f51fc/CHANGES.txt -- diff --cc CHANGES.txt index 7737269,747df0d..c4698d6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,77 -1,7 +1,78 @@@ -2.0.11: - * Explicitly disallowing mixing multi-column and single-column +2.1.1 + * (cqlsh): Support for query paging (CASSANDRA-7514) + * (cqlsh): Show progress of COPY operations (CASSANDRA-7789) + * Add syntax to remove multiple elements from a map (CASSANDRA-6599) + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839) + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606) + * (cqlsh) Display the current logged-in user (CASSANDRA-7785) + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815) + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE + output (CASSANDRA-7659) + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671) + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405) + * Support list index operations with conditions (CASSANDRA-7499) + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731) + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680) + * (cqlsh) Error when tracing query (CASSANDRA-7613) + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: + * Explicitly disallow mixing multi-column and single-column relations on clustering columns (CASSANDRA-7711) * Better error message when condition is set on PK column (CASSANDRA-7804) + * Don't send schema change responses and events for no-op DDL + statements (CASSANDRA-7600) + * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774) + * Throw InvalidRequestException when queries contain relations on entire + collection columns (CASSANDRA-7506) + * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577) + * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252) + * (cqlsh) cqlsh should automatically disable tracing when selecting + from system_traces (CASSANDRA-7641) + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) +Merged from 1.2: ++ * Don't index tombstones (CASSANDRA-7828) + * Improve PasswordAuthenticator default super user setup (CASSANDRA-7788) + + +2.1.0 + * (cqlsh) Removed "ALTER TYPE RENAME TO " from tab-completion + (CASSANDRA-7895) + * Fixed IllegalStateException in anticompaction (CASSANDRA-7892) + * cqlsh: DESCRIBE support for frozen UDTs, tuples (CASSANDRA-7863) + * Avoid exposing internal classes over JMX (CASSANDRA-7879) + * Add null check for keys when freezing collection (CASSANDRA-7869) + * Improve stress workload realism (CASSANDRA
[03/10] git commit: Don't index tombstones.
Don't index tombstones. patch by Mikhail Stepura; reviewed by Aleksey Yeschenko for CASSANDRA-7828 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6860d6fa Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6860d6fa Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6860d6fa Branch: refs/heads/cassandra-2.1 Commit: 6860d6fad9e6af90ff7c2f4359a0bdc1226135b5 Parents: 8bef260 Author: Mikhail Stepura Authored: Mon Sep 8 12:01:37 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:01:37 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6c601e5..435b124 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.2.19 + * Don't index tombstones (CASSANDRA-7828) * Don't allow compacted sstables to be marked as compacting (CASSANDRA-7145) * Track expired tombstones (CASSANDRA-7810) * Validate empty cell names from counter updates (CASSANDRA-7798) http://git-wip-us.apache.org/repos/asf/cassandra/blob/6860d6fa/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java -- diff --git a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java index 7fefa13..8839228 100644 --- a/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java +++ b/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java @@ -443,7 +443,7 @@ public class SecondaryIndexManager { for (IColumn column : cf) { -if (index.indexes(column.name())) +if (column.isLive() && index.indexes(column.name())) ((PerColumnSecondaryIndex) index).insert(key, column); } }
[08/10] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0
Merge branch 'cassandra-2.0' into cassandra-2.1.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ad9f51fc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ad9f51fc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ad9f51fc Branch: refs/heads/cassandra-2.1 Commit: ad9f51fc5cc87ff744eee0ceb4abb05f5f3a3e24 Parents: f1f5f5f 8aa3c3a Author: Mikhail Stepura Authored: Mon Sep 8 12:20:51 2014 -0700 Committer: Mikhail Stepura Committed: Mon Sep 8 12:20:51 2014 -0700 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ad9f51fc/CHANGES.txt -- diff --cc CHANGES.txt index 7737269,747df0d..c4698d6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,77 -1,7 +1,78 @@@ -2.0.11: - * Explicitly disallowing mixing multi-column and single-column +2.1.1 + * (cqlsh): Support for query paging (CASSANDRA-7514) + * (cqlsh): Show progress of COPY operations (CASSANDRA-7789) + * Add syntax to remove multiple elements from a map (CASSANDRA-6599) + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839) + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606) + * (cqlsh) Display the current logged-in user (CASSANDRA-7785) + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815) + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE + output (CASSANDRA-7659) + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671) + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405) + * Support list index operations with conditions (CASSANDRA-7499) + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731) + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680) + * (cqlsh) Error when tracing query (CASSANDRA-7613) + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: + * Explicitly disallow mixing multi-column and single-column relations on clustering columns (CASSANDRA-7711) * Better error message when condition is set on PK column (CASSANDRA-7804) + * Don't send schema change responses and events for no-op DDL + statements (CASSANDRA-7600) + * (Hadoop) fix cluster initialisation for a split fetching (CASSANDRA-7774) + * Throw InvalidRequestException when queries contain relations on entire + collection columns (CASSANDRA-7506) + * (cqlsh) enable CTRL-R history search with libedit (CASSANDRA-7577) + * (Hadoop) allow ACFRW to limit nodes to local DC (CASSANDRA-7252) + * (cqlsh) cqlsh should automatically disable tracing when selecting + from system_traces (CASSANDRA-7641) + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) +Merged from 1.2: ++ * Don't index tombstones (CASSANDRA-7828) + * Improve PasswordAuthenticator default super user setup (CASSANDRA-7788) + + +2.1.0 + * (cqlsh) Removed "ALTER TYPE RENAME TO " from tab-completion + (CASSANDRA-7895) + * Fixed IllegalStateException in anticompaction (CASSANDRA-7892) + * cqlsh: DESCRIBE support for frozen UDTs, tuples (CASSANDRA-7863) + * Avoid exposing internal classes over JMX (CASSANDRA-7879) + * Add null check for keys when freezing collection (CASSANDRA-7869) + * Improve stress workload realism (C
[jira] [Commented] (CASSANDRA-7873) java.util.ConcurrentModificationException seen after selecting from system_auth
[ https://issues.apache.org/jira/browse/CASSANDRA-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125988#comment-14125988 ] Mikhail Stepura commented on CASSANDRA-7873: +1 > java.util.ConcurrentModificationException seen after selecting from > system_auth > --- > > Key: CASSANDRA-7873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7873 > Project: Cassandra > Issue Type: Bug > Environment: OSX and Ubuntu 14.04 >Reporter: Philip Thompson >Assignee: Benedict > Fix For: 3.0 > > Attachments: 7873.21.txt, 7873.trunk.txt, 7873.txt > > > The dtest auth_test.py:TestAuth.system_auth_ks_is_alterable_test is failing > on trunk only with the following stack trace: > {code} > Unexpected error in node1 node log: > ERROR [Thrift:1] 2014-09-03 15:48:08,389 CustomTThreadPoolServer.java:219 - > Error occurred during processing of message. > java.util.ConcurrentModificationException: null > at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859) > ~[na:1.7.0_65] > at java.util.ArrayList$Itr.next(ArrayList.java:831) ~[na:1.7.0_65] > at > org.apache.cassandra.service.RowDigestResolver.resolve(RowDigestResolver.java:71) > ~[main/:na] > at > org.apache.cassandra.service.RowDigestResolver.resolve(RowDigestResolver.java:28) > ~[main/:na] > at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:110) > ~[main/:na] > at > org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:144) > ~[main/:na] > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1228) > ~[main/:na] > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1154) > ~[main/:na] > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:256) > ~[main/:na] > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:212) > ~[main/:na] > at org.apache.cassandra.auth.Auth.selectUser(Auth.java:257) ~[main/:na] > at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:76) > ~[main/:na] > at org.apache.cassandra.service.ClientState.login(ClientState.java:178) > ~[main/:na] > at > org.apache.cassandra.thrift.CassandraServer.login(CassandraServer.java:1486) > ~[main/:na] > at > org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3579) > ~[thrift/:na] > at > org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3563) > ~[thrift/:na] > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > ~[libthrift-0.9.1.jar:0.9.1] > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > ~[libthrift-0.9.1.jar:0.9.1] > at > org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201) > ~[main/:na] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_65] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] > {code} > That exception is thrown when the following query is sent: > {code} > """SELECT strategy_options > FROM system.schema_keyspaces > WHERE keyspace_name = 'system_auth'""" > {code} > The test alters the RF of the system_auth keyspace, then shuts down and > restarts the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
[ https://issues.apache.org/jira/browse/CASSANDRA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-7901: --- Attachment: 7901.txt > Implement -f functionality in stop-server.bat > - > > Key: CASSANDRA-7901 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 > Project: Cassandra > Issue Type: Improvement > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7901.txt > > > Stop-server.bat lists "-f" as an argument but does not handle it inside of > stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
[ https://issues.apache.org/jira/browse/CASSANDRA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126007#comment-14126007 ] Philip Thompson commented on CASSANDRA-7901: I'm not seeing the new console output I added when running this. > Implement -f functionality in stop-server.bat > - > > Key: CASSANDRA-7901 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 > Project: Cassandra > Issue Type: Improvement > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7901.txt > > > Stop-server.bat lists "-f" as an argument but does not handle it inside of > stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
[ https://issues.apache.org/jira/browse/CASSANDRA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126007#comment-14126007 ] Philip Thompson edited comment on CASSANDRA-7901 at 9/8/14 7:58 PM: I'm not seeing the new console output I added. Is there something obvious I'm doing wrong? was (Author: philipthompson): I'm not seeing the new console output I added when running this. > Implement -f functionality in stop-server.bat > - > > Key: CASSANDRA-7901 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 > Project: Cassandra > Issue Type: Improvement > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7901.txt > > > Stop-server.bat lists "-f" as an argument but does not handle it inside of > stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7736) Clean-up, justify (and reduce) each use of @Inline
[ https://issues.apache.org/jira/browse/CASSANDRA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-7736: -- Attachment: 7736.txt Removed the non-required ones and left the ones on the ones used by the NativeCell comparators. This yield a ~5% gain for me. > Clean-up, justify (and reduce) each use of @Inline > -- > > Key: CASSANDRA-7736 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7736 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: T Jake Luciani >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7736.txt > > > \@Inline is a delicate tool, and should in all cases we've used it (and use > it in future) be accompanied by a comment justifying its use in the given > context both theoretically and, preferably, with some brief description > of/link to steps taken to demonstrate its benefit. We should aim to not use > it unless we are very confident we can do better than the normal behaviour, > as poor use can result in a polluted instruction cache, which can yield > better results in tight benchmarks, but worse results in general use. > It looks to me that we have too many uses already. I'll look over each one as > well, and we can compare notes. If there's disagreement on any use, we can > discuss, and if still there is any dissent should always err in favour of > *not* using \@Inline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
[ https://issues.apache.org/jira/browse/CASSANDRA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126015#comment-14126015 ] Philip Thompson commented on CASSANDRA-7901: http://msdn.microsoft.com/en-us/library/system.diagnostics.process.kill(v=vs.110).aspx > Implement -f functionality in stop-server.bat > - > > Key: CASSANDRA-7901 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 > Project: Cassandra > Issue Type: Improvement > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7901.txt > > > Stop-server.bat lists "-f" as an argument but does not handle it inside of > stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
[ https://issues.apache.org/jira/browse/CASSANDRA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126023#comment-14126023 ] Joshua McKenzie commented on CASSANDRA-7901: My preference would be to keep it in PowerShell scripting context. No need to delve into C#/CLR for a process kill as that's another layer of complexity. That, and I'm not sure I trust Process.Kill to be as thorough as a taskkill /F. Most of the process killing API's try to be graceful and send a WM_CLOSE on the Windows side. > Implement -f functionality in stop-server.bat > - > > Key: CASSANDRA-7901 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 > Project: Cassandra > Issue Type: Improvement > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7901.txt > > > Stop-server.bat lists "-f" as an argument but does not handle it inside of > stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7520) Permit sorting sstables by raw partition key, as opposed to token
[ https://issues.apache.org/jira/browse/CASSANDRA-7520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126027#comment-14126027 ] Dan Hendry commented on CASSANDRA-7520: --- CASSANDRA-7890 Observes that LCS becomes incredibly efficient, to the point of reducing compaction overhead to near zero, if data on disk is sorted in the order it is inserted. This effect applies naturally to time series data but might also be beneficial to other types of workloads - MapReduce style processing and bulk loading (where the user can pre-sort their input data) both jump to mind. I for one would make the tradeoff (improved compaction and read performance at the expense of repair and bootstrapping) in a heartbeat, without reservation, for a number of our clusters. More generally, many of our workloads and data models could be changed fairly easily to take advantage of these read and compaction wins. > Permit sorting sstables by raw partition key, as opposed to token > - > > Key: CASSANDRA-7520 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7520 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > > At the moment we have some counter-intuitive behaviour, which is that with a > hashed partitioner (recommended) the more compacted the data is, the more > randomly distributed it is amongst the file. This means that data access > locality is made pretty much as bad as possible, and we rely on the OS to do > its best to fix that for us with its page cache. > [~jasobrown] mentioned this at the NGCC, but thinking on it some more it > seems that many use cases may benefit from dropping the token at the storage > level and sorting based on the raw key data. For workloads where nearness of > key => likelihood of being coreferenced, this could improve data locality and > cache hit rate dramatically. Timeseries workloads spring to mind, but I doubt > this is constrained to them. Most likely any non-random access pattern could > benefit. A random access pattern would most likely suffer from this scheme, > as we can index more efficiently into the hashed data. However there's no > reason we could not support both schemes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
[ https://issues.apache.org/jira/browse/CASSANDRA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-7901: --- Attachment: 7901-v2.txt > Implement -f functionality in stop-server.bat > - > > Key: CASSANDRA-7901 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 > Project: Cassandra > Issue Type: Improvement > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7901-v2.txt, 7901.txt > > > Stop-server.bat lists "-f" as an argument but does not handle it inside of > stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
git commit: Implement -f functionality in stop-server.bat
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 ad9f51fc5 -> c1c337536 Implement -f functionality in stop-server.bat patch by Philip Thompson; reviewed by Josh McKenzie for CASSANDRA-7901 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c1c33753 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c1c33753 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c1c33753 Branch: refs/heads/cassandra-2.1 Commit: c1c3375361eac8de2f9da24663537ed9950b11a0 Parents: ad9f51f Author: Joshua McKenzie Authored: Mon Sep 8 15:30:20 2014 -0500 Committer: Joshua McKenzie Committed: Mon Sep 8 15:30:20 2014 -0500 -- bin/stop-server.ps1 | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c1c33753/bin/stop-server.ps1 -- diff --git a/bin/stop-server.ps1 b/bin/stop-server.ps1 index d8ac970..0d125dc 100644 --- a/bin/stop-server.ps1 +++ b/bin/stop-server.ps1 @@ -161,7 +161,7 @@ Function KillProcess $env:TMP = $oldTmp $env:TEMP = $oldTemp -$a = Get-Content $p +$pidToKill = Get-Content $p # If run in cygwin, we don't get the TITLE / pid combo in stop-server.bat but also don't need # to worry about reattaching console output as it gets stderr/stdout even after the C#/C++ # FreeConsole calls. @@ -170,7 +170,14 @@ Function KillProcess $batchpid = -1 } -[PowerStopper.Stopper]::StopProgram($a, $batchpid, $silent) +if ($f) +{ +taskkill /f /pid $pidToKill +} +else +{ +[PowerStopper.Stopper]::StopProgram($pidToKill, $batchpid, $silent) +} } #-
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6cb1f077 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6cb1f077 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6cb1f077 Branch: refs/heads/trunk Commit: 6cb1f0779a981a9743e461d7626a4049a3d7faad Parents: 9288159 c1c3375 Author: Joshua McKenzie Authored: Mon Sep 8 15:32:36 2014 -0500 Committer: Joshua McKenzie Committed: Mon Sep 8 15:32:36 2014 -0500 -- bin/stop-server.ps1 | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) --
[1/2] git commit: Implement -f functionality in stop-server.bat
Repository: cassandra Updated Branches: refs/heads/trunk 928815989 -> 6cb1f0779 Implement -f functionality in stop-server.bat patch by Philip Thompson; reviewed by Josh McKenzie for CASSANDRA-7901 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c1c33753 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c1c33753 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c1c33753 Branch: refs/heads/trunk Commit: c1c3375361eac8de2f9da24663537ed9950b11a0 Parents: ad9f51f Author: Joshua McKenzie Authored: Mon Sep 8 15:30:20 2014 -0500 Committer: Joshua McKenzie Committed: Mon Sep 8 15:30:20 2014 -0500 -- bin/stop-server.ps1 | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c1c33753/bin/stop-server.ps1 -- diff --git a/bin/stop-server.ps1 b/bin/stop-server.ps1 index d8ac970..0d125dc 100644 --- a/bin/stop-server.ps1 +++ b/bin/stop-server.ps1 @@ -161,7 +161,7 @@ Function KillProcess $env:TMP = $oldTmp $env:TEMP = $oldTemp -$a = Get-Content $p +$pidToKill = Get-Content $p # If run in cygwin, we don't get the TITLE / pid combo in stop-server.bat but also don't need # to worry about reattaching console output as it gets stderr/stdout even after the C#/C++ # FreeConsole calls. @@ -170,7 +170,14 @@ Function KillProcess $batchpid = -1 } -[PowerStopper.Stopper]::StopProgram($a, $batchpid, $silent) +if ($f) +{ +taskkill /f /pid $pidToKill +} +else +{ +[PowerStopper.Stopper]::StopProgram($pidToKill, $batchpid, $silent) +} } #-
[jira] [Comment Edited] (CASSANDRA-7901) Implement -f functionality in stop-server.bat
[ https://issues.apache.org/jira/browse/CASSANDRA-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126052#comment-14126052 ] Joshua McKenzie edited comment on CASSANDRA-7901 at 9/8/14 8:34 PM: v2 LGTM and tests out cleanly. Committed. (as an aside, I did have to unix2dos the file to get v2 to apply) was (Author: joshuamckenzie): v2 LGTM and tests out cleanly. Committed. > Implement -f functionality in stop-server.bat > - > > Key: CASSANDRA-7901 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7901 > Project: Cassandra > Issue Type: Improvement > Environment: Windows >Reporter: Philip Thompson >Assignee: Philip Thompson >Priority: Minor > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7901-v2.txt, 7901.txt > > > Stop-server.bat lists "-f" as an argument but does not handle it inside of > stop-server.ps1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7898) IndexOutOfBoundsException comparing incompatible reversed types in DynamicCompositeType
[ https://issues.apache.org/jira/browse/CASSANDRA-7898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Whittington updated CASSANDRA-7898: --- Attachment: cassandra-2.0-7898.txt Original patch lost ordering of compatible ReversedType(X) in a dynamic composite - updated patch fixes this and adds additional testing for preservation of order in reversed types. > IndexOutOfBoundsException comparing incompatible reversed types in > DynamicCompositeType > --- > > Key: CASSANDRA-7898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7898 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Tim Whittington > Attachments: cassandra-2.0-7898.txt, cassandra-2.0-7898.txt > > > In the following setup, an IndexOutOfBoundsException is often observed: > * A DynamicCompositeType value is used in a column name (e.g. in a CQL3 > cluster key) > * Two incompatible types (e.g. Int32Type and DoubleType) are used in the > dynamic composite value in their reversed form (e.g. ReversedType(Int32Type) > and ReversedType(DoubleType) > * Values for the incompatible types are inserted > * One of various scenarios occurs that trigger comparison of the column names > The exception can be variously observed (sometimes not immediately) during > query execution, memtable flushing, commit log replay etc. In some cases this > can prevent Cassandra server startup (e.g. during commit log replay). > Typical stack traces follow: > {noformat} > java.lang.RuntimeException: java.lang.IndexOutOfBoundsException > at > org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:1968) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.IndexOutOfBoundsException > at java.nio.Buffer.checkIndex(Buffer.java:538) > at java.nio.HeapByteBuffer.getDouble(HeapByteBuffer.java:512) > at > org.apache.cassandra.utils.ByteBufferUtil.toDouble(ByteBufferUtil.java:431) > at > org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:33) > at > org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:25) > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:142) > at > org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:45) > at > org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:28) > at > org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:73) > at > org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:30) > at > org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at > org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at > edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059) > at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023) > at > edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985) > at > org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:319) > at > org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:191) > at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226) > at org.apache.cassandra.db.Memtable.put(Memtable.java:173) > at > org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:900) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:374) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) > at org.apache.cassandra.db.RowMutation.apply(R
[jira] [Updated] (CASSANDRA-7898) IndexOutOfBoundsException comparing incompatible reversed types in DynamicCompositeType
[ https://issues.apache.org/jira/browse/CASSANDRA-7898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Whittington updated CASSANDRA-7898: --- Attachment: (was: cassandra-2.0-7898.txt) > IndexOutOfBoundsException comparing incompatible reversed types in > DynamicCompositeType > --- > > Key: CASSANDRA-7898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7898 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Tim Whittington > Attachments: cassandra-2.0-7898.txt > > > In the following setup, an IndexOutOfBoundsException is often observed: > * A DynamicCompositeType value is used in a column name (e.g. in a CQL3 > cluster key) > * Two incompatible types (e.g. Int32Type and DoubleType) are used in the > dynamic composite value in their reversed form (e.g. ReversedType(Int32Type) > and ReversedType(DoubleType) > * Values for the incompatible types are inserted > * One of various scenarios occurs that trigger comparison of the column names > The exception can be variously observed (sometimes not immediately) during > query execution, memtable flushing, commit log replay etc. In some cases this > can prevent Cassandra server startup (e.g. during commit log replay). > Typical stack traces follow: > {noformat} > java.lang.RuntimeException: java.lang.IndexOutOfBoundsException > at > org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:1968) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.IndexOutOfBoundsException > at java.nio.Buffer.checkIndex(Buffer.java:538) > at java.nio.HeapByteBuffer.getDouble(HeapByteBuffer.java:512) > at > org.apache.cassandra.utils.ByteBufferUtil.toDouble(ByteBufferUtil.java:431) > at > org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:33) > at > org.apache.cassandra.serializers.DoubleSerializer.deserialize(DoubleSerializer.java:25) > at > org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:142) > at > org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:45) > at > org.apache.cassandra.db.marshal.DoubleType.compare(DoubleType.java:28) > at > org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:73) > at > org.apache.cassandra.db.marshal.ReversedType.compare(ReversedType.java:30) > at > org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at > org.apache.cassandra.db.marshal.AbstractType.compareCollectionMembers(AbstractType.java:279) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:64) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:36) > at > edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1192) > at > edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059) > at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023) > at > edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985) > at > org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:319) > at > org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:191) > at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226) > at org.apache.cassandra.db.Memtable.put(Memtable.java:173) > at > org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:900) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:374) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) > at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211) > at > org.apache.cassandra.service.StorageProxy$7.runMayThrow(StorageProxy.java:952) > at > org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(St
[jira] [Created] (CASSANDRA-7902) Introduce the ability to ignore RR based on consistencfy
Brandon Williams created CASSANDRA-7902: --- Summary: Introduce the ability to ignore RR based on consistencfy Key: CASSANDRA-7902 URL: https://issues.apache.org/jira/browse/CASSANDRA-7902 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Brandon Williams There exists a case for LOCAL_* consistency levels where you really want them *local only*. This implies that you don't ever want to do cross-dc RR, but do for other levels. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7895) ALTER TYPE RENAME TO no longer parses as valid cql
[ https://issues.apache.org/jira/browse/CASSANDRA-7895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126318#comment-14126318 ] Russ Hatch commented on CASSANDRA-7895: --- dtests should be updated shortly to no longer attempt UDT rename and to add some field rename coverage. > ALTER TYPE RENAME TO no longer parses as valid cql > > > Key: CASSANDRA-7895 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7895 > Project: Cassandra > Issue Type: Bug >Reporter: Russ Hatch >Assignee: Mikhail Stepura >Priority: Minor > Labels: qa-resolved > Fix For: 2.1.0, 2.1.1 > > Attachments: CASSANDRA-2.1.0-7895.patch > > > Type renaming seems to be broken. The error looks like perhaps the syntax has > changed or there's a problem parsing the cql. > {noformat} > cqlsh:test> create type foo (somefield int); > cqlsh:test> alter type foo rename to bar; > > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7736) Clean-up, justify (and reduce) each use of @Inline
[ https://issues.apache.org/jira/browse/CASSANDRA-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126399#comment-14126399 ] Benedict commented on CASSANDRA-7736: - LGTM +1 > Clean-up, justify (and reduce) each use of @Inline > -- > > Key: CASSANDRA-7736 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7736 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: T Jake Luciani >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7736.txt > > > \@Inline is a delicate tool, and should in all cases we've used it (and use > it in future) be accompanied by a comment justifying its use in the given > context both theoretically and, preferably, with some brief description > of/link to steps taken to demonstrate its benefit. We should aim to not use > it unless we are very confident we can do better than the normal behaviour, > as poor use can result in a polluted instruction cache, which can yield > better results in tight benchmarks, but worse results in general use. > It looks to me that we have too many uses already. I'll look over each one as > well, and we can compare notes. If there's disagreement on any use, we can > discuss, and if still there is any dissent should always err in favour of > *not* using \@Inline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-7903) tombstone created upon insert of new row
Thanh created CASSANDRA-7903: Summary: tombstone created upon insert of new row Key: CASSANDRA-7903 URL: https://issues.apache.org/jira/browse/CASSANDRA-7903 Project: Cassandra Issue Type: Bug Reporter: Thanh Tombstone is created upon insert of new row, depending on how the row is inserted. Simple way to observe this behavior: Using cqlsh: CREATE TABLE users1 ( userid text PRIMARY KEY, first_name text, last_name text); insert into users1 (userid, first_name) values ('a','a'); tracing on; select * from users; Trace results show 1 live cell and 0 tombstone cells created as a result: userid | first_name | last_name ++--- a | a | null (1 rows) … Read 1 live and 0 tombstoned cells | 00:31:31,487 | 10.240.203.201 | 1275 Scanned 1 rows and matched 1 | 00:31:31,487 | 10.240.203.201 | 1328 … Now, insert into users1 (userid, first_name,last_name) values ('b','b',null); select * from users; Trace results show 1 live cell and 1 tombstone cell created as a result: userid | first_name | last_name ++--- a | a | null b | b | null (2 rows) … Read 1 live and 0 tombstoned cells | 00:35:09,357 | 10.240.203.201 | 1243 Read 1 live and 1 tombstoned cells | 00:35:09,357 | 10.240.203.201 | 1383 Scanned 2 rows and matched 2 | 00:35:09,357 | 10.240.203.201 | 1438 … Tombstone is not expected to be created in either case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
git commit: fix stress metrics format strings
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1.0 0580fb2b7 -> 852f084f0 fix stress metrics format strings Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/852f084f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/852f084f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/852f084f Branch: refs/heads/cassandra-2.1.0 Commit: 852f084f03f8b71b2275fd2e51fd0f2c08e0a0ab Parents: 0580fb2 Author: Dave Brosius Authored: Mon Sep 8 22:20:29 2014 -0400 Committer: Dave Brosius Committed: Mon Sep 8 22:20:29 2014 -0400 -- tools/stress/src/org/apache/cassandra/stress/StressMetrics.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/852f084f/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java -- diff --git a/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java b/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java index a9edfc6..dd3b867 100644 --- a/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java +++ b/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java @@ -132,8 +132,8 @@ public class StressMetrics // PRINT FORMATTING -public static final String HEADFORMAT = "%-10s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%7s,%9s"; -public static final String ROWFORMAT = "%-10d,%8.0f,%8.0f,%8.0f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%7.1f,%9.5f"; +public static final String HEADFORMAT = "%-10s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%7s,%9s"; +public static final String ROWFORMAT = "%-10d,%8.0f,%8.0f,%8.0f,%8.0f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%7.1f,%9.5f"; private static void printHeader(String prefix, PrintStream output) {
[1/2] git commit: fix stress metrics format strings
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 c1c337536 -> 4bdbdc1e3 fix stress metrics format strings Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/852f084f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/852f084f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/852f084f Branch: refs/heads/cassandra-2.1 Commit: 852f084f03f8b71b2275fd2e51fd0f2c08e0a0ab Parents: 0580fb2 Author: Dave Brosius Authored: Mon Sep 8 22:20:29 2014 -0400 Committer: Dave Brosius Committed: Mon Sep 8 22:20:29 2014 -0400 -- tools/stress/src/org/apache/cassandra/stress/StressMetrics.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/852f084f/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java -- diff --git a/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java b/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java index a9edfc6..dd3b867 100644 --- a/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java +++ b/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java @@ -132,8 +132,8 @@ public class StressMetrics // PRINT FORMATTING -public static final String HEADFORMAT = "%-10s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%7s,%9s"; -public static final String ROWFORMAT = "%-10d,%8.0f,%8.0f,%8.0f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%7.1f,%9.5f"; +public static final String HEADFORMAT = "%-10s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%7s,%9s"; +public static final String ROWFORMAT = "%-10d,%8.0f,%8.0f,%8.0f,%8.0f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%7.1f,%9.5f"; private static void printHeader(String prefix, PrintStream output) {
[2/3] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4bdbdc1e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4bdbdc1e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4bdbdc1e Branch: refs/heads/trunk Commit: 4bdbdc1e397902ff35623878c7264324bf675afd Parents: c1c3375 852f084 Author: Dave Brosius Authored: Mon Sep 8 22:21:13 2014 -0400 Committer: Dave Brosius Committed: Mon Sep 8 22:21:13 2014 -0400 -- tools/stress/src/org/apache/cassandra/stress/StressMetrics.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --
[2/2] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4bdbdc1e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4bdbdc1e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4bdbdc1e Branch: refs/heads/cassandra-2.1 Commit: 4bdbdc1e397902ff35623878c7264324bf675afd Parents: c1c3375 852f084 Author: Dave Brosius Authored: Mon Sep 8 22:21:13 2014 -0400 Committer: Dave Brosius Committed: Mon Sep 8 22:21:13 2014 -0400 -- tools/stress/src/org/apache/cassandra/stress/StressMetrics.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1198df4b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1198df4b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1198df4b Branch: refs/heads/trunk Commit: 1198df4beb50c03a7900ff55cd11dde8aff6c377 Parents: 6cb1f07 4bdbdc1 Author: Dave Brosius Authored: Mon Sep 8 22:21:59 2014 -0400 Committer: Dave Brosius Committed: Mon Sep 8 22:21:59 2014 -0400 -- tools/stress/src/org/apache/cassandra/stress/StressMetrics.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1198df4b/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java --
[1/3] git commit: fix stress metrics format strings
Repository: cassandra Updated Branches: refs/heads/trunk 6cb1f0779 -> 1198df4be fix stress metrics format strings Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/852f084f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/852f084f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/852f084f Branch: refs/heads/trunk Commit: 852f084f03f8b71b2275fd2e51fd0f2c08e0a0ab Parents: 0580fb2 Author: Dave Brosius Authored: Mon Sep 8 22:20:29 2014 -0400 Committer: Dave Brosius Committed: Mon Sep 8 22:20:29 2014 -0400 -- tools/stress/src/org/apache/cassandra/stress/StressMetrics.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/852f084f/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java -- diff --git a/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java b/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java index a9edfc6..dd3b867 100644 --- a/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java +++ b/tools/stress/src/org/apache/cassandra/stress/StressMetrics.java @@ -132,8 +132,8 @@ public class StressMetrics // PRINT FORMATTING -public static final String HEADFORMAT = "%-10s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%7s,%9s"; -public static final String ROWFORMAT = "%-10d,%8.0f,%8.0f,%8.0f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%7.1f,%9.5f"; +public static final String HEADFORMAT = "%-10s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%8s,%7s,%9s"; +public static final String ROWFORMAT = "%-10d,%8.0f,%8.0f,%8.0f,%8.0f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%8.1f,%7.1f,%9.5f"; private static void printHeader(String prefix, PrintStream output) {
[1/6] git commit: In RowDataResolver, reduce calls to replies.size(), and use replies.peek() instead of replies.iterator().next()
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 4bdbdc1e3 -> bbbdec63a refs/heads/cassandra-2.1.0 852f084f0 -> f099e086f refs/heads/trunk 1198df4be -> 691d5308a In RowDataResolver, reduce calls to replies.size(), and use replies.peek() instead of replies.iterator().next() Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f099e086 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f099e086 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f099e086 Branch: refs/heads/cassandra-2.1 Commit: f099e086f3f002789e24bd6c58e52b7553cd5381 Parents: 852f084 Author: Benedict Elliott Smith Authored: Tue Sep 9 09:36:14 2014 +0700 Committer: Benedict Elliott Smith Committed: Tue Sep 9 09:38:16 2014 +0700 -- .../apache/cassandra/service/AbstractRowResolver.java| 4 ++-- .../org/apache/cassandra/service/RowDataResolver.java| 11 ++- 2 files changed, 8 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f099e086/src/java/org/apache/cassandra/service/AbstractRowResolver.java -- diff --git a/src/java/org/apache/cassandra/service/AbstractRowResolver.java b/src/java/org/apache/cassandra/service/AbstractRowResolver.java index 1fbb92b..fbbf473 100644 --- a/src/java/org/apache/cassandra/service/AbstractRowResolver.java +++ b/src/java/org/apache/cassandra/service/AbstractRowResolver.java @@ -18,7 +18,7 @@ package org.apache.cassandra.service; import java.nio.ByteBuffer; -import java.util.Collection; +import java.util.Queue; import java.util.concurrent.ConcurrentLinkedQueue; import org.slf4j.Logger; @@ -35,7 +35,7 @@ public abstract class AbstractRowResolver implements IResponseResolver> replies = new ConcurrentLinkedQueue<>(); +protected final Queue> replies = new ConcurrentLinkedQueue<>(); protected final DecoratedKey key; public AbstractRowResolver(ByteBuffer key, String keyspaceName) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f099e086/src/java/org/apache/cassandra/service/RowDataResolver.java -- diff --git a/src/java/org/apache/cassandra/service/RowDataResolver.java b/src/java/org/apache/cassandra/service/RowDataResolver.java index f5eee40..9c24776 100644 --- a/src/java/org/apache/cassandra/service/RowDataResolver.java +++ b/src/java/org/apache/cassandra/service/RowDataResolver.java @@ -57,15 +57,16 @@ public class RowDataResolver extends AbstractRowResolver */ public Row resolve() throws DigestMismatchException { +int replyCount = replies.size(); if (logger.isDebugEnabled()) -logger.debug("resolving {} responses", replies.size()); +logger.debug("resolving {} responses", replyCount); long start = System.nanoTime(); ColumnFamily resolved; -if (replies.size() > 1) +if (replyCount > 1) { -List versions = new ArrayList(replies.size()); -List endpoints = new ArrayList(replies.size()); +List versions = new ArrayList<>(replyCount); +List endpoints = new ArrayList<>(replyCount); for (MessageIn message : replies) { @@ -158,7 +159,7 @@ public class RowDataResolver extends AbstractRowResolver public Row getData() { -return replies.iterator().next().payload.row(); +return replies.peek().payload.row(); } public boolean isDataPresent()
[4/6] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bbbdec63 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bbbdec63 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bbbdec63 Branch: refs/heads/trunk Commit: bbbdec63a419018cfb77732a55c66e140e7ce955 Parents: 4bdbdc1 f099e08 Author: Benedict Elliott Smith Authored: Tue Sep 9 09:38:29 2014 +0700 Committer: Benedict Elliott Smith Committed: Tue Sep 9 09:38:29 2014 +0700 -- .../apache/cassandra/service/AbstractRowResolver.java| 4 ++-- .../org/apache/cassandra/service/RowDataResolver.java| 11 ++- 2 files changed, 8 insertions(+), 7 deletions(-) --
[6/6] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/service/AbstractRowResolver.java src/java/org/apache/cassandra/service/RowDataResolver.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/691d5308 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/691d5308 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/691d5308 Branch: refs/heads/trunk Commit: 691d5308a7b07418cef5eee7f06f0ca03f639a4b Parents: 1198df4 bbbdec6 Author: Benedict Elliott Smith Authored: Tue Sep 9 09:39:26 2014 +0700 Committer: Benedict Elliott Smith Committed: Tue Sep 9 09:39:26 2014 +0700 -- src/java/org/apache/cassandra/service/RowDataResolver.java | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/691d5308/src/java/org/apache/cassandra/service/RowDataResolver.java --
[5/6] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bbbdec63 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bbbdec63 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bbbdec63 Branch: refs/heads/cassandra-2.1 Commit: bbbdec63a419018cfb77732a55c66e140e7ce955 Parents: 4bdbdc1 f099e08 Author: Benedict Elliott Smith Authored: Tue Sep 9 09:38:29 2014 +0700 Committer: Benedict Elliott Smith Committed: Tue Sep 9 09:38:29 2014 +0700 -- .../apache/cassandra/service/AbstractRowResolver.java| 4 ++-- .../org/apache/cassandra/service/RowDataResolver.java| 11 ++- 2 files changed, 8 insertions(+), 7 deletions(-) --
[2/6] git commit: In RowDataResolver, reduce calls to replies.size(), and use replies.peek() instead of replies.iterator().next()
In RowDataResolver, reduce calls to replies.size(), and use replies.peek() instead of replies.iterator().next() Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f099e086 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f099e086 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f099e086 Branch: refs/heads/cassandra-2.1.0 Commit: f099e086f3f002789e24bd6c58e52b7553cd5381 Parents: 852f084 Author: Benedict Elliott Smith Authored: Tue Sep 9 09:36:14 2014 +0700 Committer: Benedict Elliott Smith Committed: Tue Sep 9 09:38:16 2014 +0700 -- .../apache/cassandra/service/AbstractRowResolver.java| 4 ++-- .../org/apache/cassandra/service/RowDataResolver.java| 11 ++- 2 files changed, 8 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f099e086/src/java/org/apache/cassandra/service/AbstractRowResolver.java -- diff --git a/src/java/org/apache/cassandra/service/AbstractRowResolver.java b/src/java/org/apache/cassandra/service/AbstractRowResolver.java index 1fbb92b..fbbf473 100644 --- a/src/java/org/apache/cassandra/service/AbstractRowResolver.java +++ b/src/java/org/apache/cassandra/service/AbstractRowResolver.java @@ -18,7 +18,7 @@ package org.apache.cassandra.service; import java.nio.ByteBuffer; -import java.util.Collection; +import java.util.Queue; import java.util.concurrent.ConcurrentLinkedQueue; import org.slf4j.Logger; @@ -35,7 +35,7 @@ public abstract class AbstractRowResolver implements IResponseResolver> replies = new ConcurrentLinkedQueue<>(); +protected final Queue> replies = new ConcurrentLinkedQueue<>(); protected final DecoratedKey key; public AbstractRowResolver(ByteBuffer key, String keyspaceName) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f099e086/src/java/org/apache/cassandra/service/RowDataResolver.java -- diff --git a/src/java/org/apache/cassandra/service/RowDataResolver.java b/src/java/org/apache/cassandra/service/RowDataResolver.java index f5eee40..9c24776 100644 --- a/src/java/org/apache/cassandra/service/RowDataResolver.java +++ b/src/java/org/apache/cassandra/service/RowDataResolver.java @@ -57,15 +57,16 @@ public class RowDataResolver extends AbstractRowResolver */ public Row resolve() throws DigestMismatchException { +int replyCount = replies.size(); if (logger.isDebugEnabled()) -logger.debug("resolving {} responses", replies.size()); +logger.debug("resolving {} responses", replyCount); long start = System.nanoTime(); ColumnFamily resolved; -if (replies.size() > 1) +if (replyCount > 1) { -List versions = new ArrayList(replies.size()); -List endpoints = new ArrayList(replies.size()); +List versions = new ArrayList<>(replyCount); +List endpoints = new ArrayList<>(replyCount); for (MessageIn message : replies) { @@ -158,7 +159,7 @@ public class RowDataResolver extends AbstractRowResolver public Row getData() { -return replies.iterator().next().payload.row(); +return replies.peek().payload.row(); } public boolean isDataPresent()
[3/6] git commit: In RowDataResolver, reduce calls to replies.size(), and use replies.peek() instead of replies.iterator().next()
In RowDataResolver, reduce calls to replies.size(), and use replies.peek() instead of replies.iterator().next() Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f099e086 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f099e086 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f099e086 Branch: refs/heads/trunk Commit: f099e086f3f002789e24bd6c58e52b7553cd5381 Parents: 852f084 Author: Benedict Elliott Smith Authored: Tue Sep 9 09:36:14 2014 +0700 Committer: Benedict Elliott Smith Committed: Tue Sep 9 09:38:16 2014 +0700 -- .../apache/cassandra/service/AbstractRowResolver.java| 4 ++-- .../org/apache/cassandra/service/RowDataResolver.java| 11 ++- 2 files changed, 8 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f099e086/src/java/org/apache/cassandra/service/AbstractRowResolver.java -- diff --git a/src/java/org/apache/cassandra/service/AbstractRowResolver.java b/src/java/org/apache/cassandra/service/AbstractRowResolver.java index 1fbb92b..fbbf473 100644 --- a/src/java/org/apache/cassandra/service/AbstractRowResolver.java +++ b/src/java/org/apache/cassandra/service/AbstractRowResolver.java @@ -18,7 +18,7 @@ package org.apache.cassandra.service; import java.nio.ByteBuffer; -import java.util.Collection; +import java.util.Queue; import java.util.concurrent.ConcurrentLinkedQueue; import org.slf4j.Logger; @@ -35,7 +35,7 @@ public abstract class AbstractRowResolver implements IResponseResolver> replies = new ConcurrentLinkedQueue<>(); +protected final Queue> replies = new ConcurrentLinkedQueue<>(); protected final DecoratedKey key; public AbstractRowResolver(ByteBuffer key, String keyspaceName) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f099e086/src/java/org/apache/cassandra/service/RowDataResolver.java -- diff --git a/src/java/org/apache/cassandra/service/RowDataResolver.java b/src/java/org/apache/cassandra/service/RowDataResolver.java index f5eee40..9c24776 100644 --- a/src/java/org/apache/cassandra/service/RowDataResolver.java +++ b/src/java/org/apache/cassandra/service/RowDataResolver.java @@ -57,15 +57,16 @@ public class RowDataResolver extends AbstractRowResolver */ public Row resolve() throws DigestMismatchException { +int replyCount = replies.size(); if (logger.isDebugEnabled()) -logger.debug("resolving {} responses", replies.size()); +logger.debug("resolving {} responses", replyCount); long start = System.nanoTime(); ColumnFamily resolved; -if (replies.size() > 1) +if (replyCount > 1) { -List versions = new ArrayList(replies.size()); -List endpoints = new ArrayList(replies.size()); +List versions = new ArrayList<>(replyCount); +List endpoints = new ArrayList<>(replyCount); for (MessageIn message : replies) { @@ -158,7 +159,7 @@ public class RowDataResolver extends AbstractRowResolver public Row getData() { -return replies.iterator().next().payload.row(); +return replies.peek().payload.row(); } public boolean isDataPresent()
git commit: Introduce new append-only concurrent collection, Accumulator, and use for AbstractRowResolver.replies
Repository: cassandra Updated Branches: refs/heads/trunk 691d5308a -> b86705027 Introduce new append-only concurrent collection, Accumulator, and use for AbstractRowResolver.replies patch by benedict; reviewed by mishail for CASSANDRA-7873 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8670502 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8670502 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8670502 Branch: refs/heads/trunk Commit: b867050270408ed6cc77c03d78d75cce9799e38e Parents: 691d530 Author: Benedict Elliott Smith Authored: Tue Sep 9 09:47:17 2014 +0700 Committer: Benedict Elliott Smith Committed: Tue Sep 9 09:47:17 2014 +0700 -- .../cassandra/service/AbstractReadExecutor.java | 2 +- .../cassandra/service/AbstractRowResolver.java | 11 +- .../apache/cassandra/service/ReadCallback.java | 2 +- .../cassandra/service/RowDataResolver.java | 4 +- .../cassandra/service/RowDigestResolver.java| 4 +- .../apache/cassandra/service/StorageProxy.java | 2 +- .../cassandra/utils/concurrent/Accumulator.java | 133 +++ 7 files changed, 145 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8670502/src/java/org/apache/cassandra/service/AbstractReadExecutor.java -- diff --git a/src/java/org/apache/cassandra/service/AbstractReadExecutor.java b/src/java/org/apache/cassandra/service/AbstractReadExecutor.java index 2c3261f..d08c63e 100644 --- a/src/java/org/apache/cassandra/service/AbstractReadExecutor.java +++ b/src/java/org/apache/cassandra/service/AbstractReadExecutor.java @@ -66,7 +66,7 @@ public abstract class AbstractReadExecutor { this.command = command; this.targetReplicas = targetReplicas; -resolver = new RowDigestResolver(command.ksName, command.key); +resolver = new RowDigestResolver(command.ksName, command.key, targetReplicas.size()); handler = new ReadCallback<>(resolver, consistencyLevel, command, targetReplicas); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8670502/src/java/org/apache/cassandra/service/AbstractRowResolver.java -- diff --git a/src/java/org/apache/cassandra/service/AbstractRowResolver.java b/src/java/org/apache/cassandra/service/AbstractRowResolver.java index 6db2569..f362047 100644 --- a/src/java/org/apache/cassandra/service/AbstractRowResolver.java +++ b/src/java/org/apache/cassandra/service/AbstractRowResolver.java @@ -18,9 +18,6 @@ package org.apache.cassandra.service; import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -29,20 +26,22 @@ import org.apache.cassandra.db.DecoratedKey; import org.apache.cassandra.db.ReadResponse; import org.apache.cassandra.db.Row; import org.apache.cassandra.net.MessageIn; +import org.apache.cassandra.utils.concurrent.Accumulator; public abstract class AbstractRowResolver implements IResponseResolver { protected static final Logger logger = LoggerFactory.getLogger(AbstractRowResolver.class); protected final String keyspaceName; -// synchronizedList gives us thread-safety without the overhead of guaranteeing uniqueness like a Set would -protected final List> replies = Collections.synchronizedList(new ArrayList>()); +// Accumulator gives us non-blocking thread-safety with optimal algorithmic constraints +protected final Accumulator> replies; protected final DecoratedKey key; -public AbstractRowResolver(ByteBuffer key, String keyspaceName) +public AbstractRowResolver(ByteBuffer key, String keyspaceName, int maxResponseCount) { this.key = StorageService.getPartitioner().decorateKey(key); this.keyspaceName = keyspaceName; +this.replies = new Accumulator<>(maxResponseCount); } public void preprocess(MessageIn message) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8670502/src/java/org/apache/cassandra/service/ReadCallback.java -- diff --git a/src/java/org/apache/cassandra/service/ReadCallback.java b/src/java/org/apache/cassandra/service/ReadCallback.java index 29eaadf..51e1818 100644 --- a/src/java/org/apache/cassandra/service/ReadCallback.java +++ b/src/java/org/apache/cassandra/service/ReadCallback.java @@ -185,7 +185,7 @@ public class ReadCallback implements IAsyncCallback message = ((ReadCommand) command).createMessage(); http://git-wip-us.apache.org/repos/asf/cassandra/blob/b8670502/src/java/org/apache/cassandra/service
[jira] [Updated] (CASSANDRA-7873) Replace AbstractRowResolver.replies with collection with tailored properties
[ https://issues.apache.org/jira/browse/CASSANDRA-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7873: Summary: Replace AbstractRowResolver.replies with collection with tailored properties (was: java.util.ConcurrentModificationException seen after selecting from system_auth) > Replace AbstractRowResolver.replies with collection with tailored properties > > > Key: CASSANDRA-7873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7873 > Project: Cassandra > Issue Type: Bug > Environment: OSX and Ubuntu 14.04 >Reporter: Philip Thompson >Assignee: Benedict > Fix For: 3.0 > > Attachments: 7873.21.txt, 7873.trunk.txt, 7873.txt > > > The dtest auth_test.py:TestAuth.system_auth_ks_is_alterable_test is failing > on trunk only with the following stack trace: > {code} > Unexpected error in node1 node log: > ERROR [Thrift:1] 2014-09-03 15:48:08,389 CustomTThreadPoolServer.java:219 - > Error occurred during processing of message. > java.util.ConcurrentModificationException: null > at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859) > ~[na:1.7.0_65] > at java.util.ArrayList$Itr.next(ArrayList.java:831) ~[na:1.7.0_65] > at > org.apache.cassandra.service.RowDigestResolver.resolve(RowDigestResolver.java:71) > ~[main/:na] > at > org.apache.cassandra.service.RowDigestResolver.resolve(RowDigestResolver.java:28) > ~[main/:na] > at org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:110) > ~[main/:na] > at > org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:144) > ~[main/:na] > at > org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1228) > ~[main/:na] > at > org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1154) > ~[main/:na] > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:256) > ~[main/:na] > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:212) > ~[main/:na] > at org.apache.cassandra.auth.Auth.selectUser(Auth.java:257) ~[main/:na] > at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:76) > ~[main/:na] > at org.apache.cassandra.service.ClientState.login(ClientState.java:178) > ~[main/:na] > at > org.apache.cassandra.thrift.CassandraServer.login(CassandraServer.java:1486) > ~[main/:na] > at > org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3579) > ~[thrift/:na] > at > org.apache.cassandra.thrift.Cassandra$Processor$login.getResult(Cassandra.java:3563) > ~[thrift/:na] > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > ~[libthrift-0.9.1.jar:0.9.1] > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > ~[libthrift-0.9.1.jar:0.9.1] > at > org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201) > ~[main/:na] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_65] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] > {code} > That exception is thrown when the following query is sent: > {code} > """SELECT strategy_options > FROM system.schema_keyspaces > WHERE keyspace_name = 'system_auth'""" > {code} > The test alters the RF of the system_auth keyspace, then shuts down and > restarts the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7861) Node is not able to gossip
[ https://issues.apache.org/jira/browse/CASSANDRA-7861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ananthkumar K S updated CASSANDRA-7861: --- Attachment: (was: thread-dump.txt) > Node is not able to gossip > -- > > Key: CASSANDRA-7861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7861 > Project: Cassandra > Issue Type: Bug >Reporter: Ananthkumar K S > Fix For: 2.0.3 > > > The node is running on xxx.xxx.xxx.xxx. All of a sudden, it was not able to > gossip and find the other nodes between data centres. We had two nodes > indicated as down in DC1 but those two nodes were up and running in DC2. When > we check the two nodes status in DC2, all the nodes in DC1 are denoted as DN > and the other node in DC2 is denoted as down. > There seems to be a disconnect between the nodes. I have attached the thread > dump of the node that was down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)