[jira] [Commented] (CASSANDRA-8108) ClassCastException in AbstractCellNameType
[ https://issues.apache.org/jira/browse/CASSANDRA-8108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168972#comment-14168972 ] David Hearnden commented on CASSANDRA-8108: --- I get this performing a `SELECT DISCTINCT` query on some static columns using the Java driver version 2.1.1 > ClassCastException in AbstractCellNameType > -- > > Key: CASSANDRA-8108 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8108 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 2.1.0 > Datastax AMI on EC2 (Ubuntu Linux) >Reporter: David Hearnden > > java.lang.ClassCastException: > org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast > to org.apache.cassandra.db.composites.CellName > at > org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.service.pager.SliceQueryPager.(SliceQueryPager.java:57) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.service.pager.MultiPartitionPager.(MultiPartitionPager.java:68) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133) > ~[apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422) > [apache-cassandra-2.1.0.jar:2.1.0] > at > org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318) > [apache-cassandra-2.1.0.jar:2.1.0] > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103) > [netty-all-4.0.20.Final.jar:4.0.20.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332) > [netty-all-4.0.20.Final.jar:4.0.20.Final] > at > io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31) > [netty-all-4.0.20.Final.jar:4.0.20.Final] > at > io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323) > [netty-all-4.0.20.Final.jar:4.0.20.Final] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_51] > at > org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163) > [apache-cassandra-2.1.0.jar:2.1.0] > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) > [apache-cassandra-2.1.0.jar:2.1.0] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8107) ClassCastException in AbstractCellNameType
[ https://issues.apache.org/jira/browse/CASSANDRA-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Hearnden resolved CASSANDRA-8107. --- Resolution: Duplicate > ClassCastException in AbstractCellNameType > -- > > Key: CASSANDRA-8107 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8107 > Project: Cassandra > Issue Type: Bug >Reporter: David Hearnden > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8108) ClassCastException in AbstractCellNameType
David Hearnden created CASSANDRA-8108: - Summary: ClassCastException in AbstractCellNameType Key: CASSANDRA-8108 URL: https://issues.apache.org/jira/browse/CASSANDRA-8108 Project: Cassandra Issue Type: Bug Environment: Cassandra 2.1.0 Datastax AMI on EC2 (Ubuntu Linux) Reporter: David Hearnden java.lang.ClassCastException: org.apache.cassandra.db.composites.Composites$EmptyComposite cannot be cast to org.apache.cassandra.db.composites.CellName at org.apache.cassandra.db.composites.AbstractCellNameType.cellFromByteBuffer(AbstractCellNameType.java:170) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.service.pager.SliceQueryPager.(SliceQueryPager.java:57) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.service.pager.MultiPartitionPager.makePager(MultiPartitionPager.java:84) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.service.pager.MultiPartitionPager.(MultiPartitionPager.java:68) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:101) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.service.pager.QueryPagers.pager(QueryPagers.java:125) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:215) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:60) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:187) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:413) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:133) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:422) [apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:318) [apache-cassandra-2.1.0.jar:2.1.0] at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103) [netty-all-4.0.20.Final.jar:4.0.20.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332) [netty-all-4.0.20.Final.jar:4.0.20.Final] at io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31) [netty-all-4.0.20.Final.jar:4.0.20.Final] at io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323) [netty-all-4.0.20.Final.jar:4.0.20.Final] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51] at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163) [apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) [apache-cassandra-2.1.0.jar:2.1.0] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8107) ClassCastException in AbstractCellNameType
David Hearnden created CASSANDRA-8107: - Summary: ClassCastException in AbstractCellNameType Key: CASSANDRA-8107 URL: https://issues.apache.org/jira/browse/CASSANDRA-8107 Project: Cassandra Issue Type: Bug Reporter: David Hearnden -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7949) LCS compaction low performance, many pending compactions, nodes are almost idle
[ https://issues.apache.org/jira/browse/CASSANDRA-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168822#comment-14168822 ] Nikolai Grigoriev edited comment on CASSANDRA-7949 at 10/12/14 11:59 PM: - I did another round of testing and I can confirm my previous suspicion. If LCS goes into "STCS fallback" mode there seems to be some kind of "point of no return". After loading fairly large amount of data I end up with a number of large (from few Gb to 200+Gb) sstables. After that the cluster simply goes downhill - it never recovers. Even if there is no traffic except the repair service (DSE OpsCenter) the number of pending compactions never declines. It actually grows. Sstables also grow and grow in size until the moment one of the compactions runs out of disk space and crashes the node. Also I believe once in this state there is no way out. sstablesplit tool, as far as I understand, cannot be used with the live node. And the tool splits the data in single thread. I have measured its performance on my system, it processes about 13Mb/s on average, thus, to split all these large sstables it would take many DAYS. I have got an idea that might actually help. That JVM property from CASSANDRA-6621 - it seems to be what I need right now. I have tried it and it seems (so far) that when compacting my nodes produce only the sstables of the target size, i.e (I may be wrong but so far it seems so) it is splitting the large sstables into the small ones while the nodes are on. If it continues like this I may hope to eventually get rid of mega-huge-sstables and then LCS performance should be back to normal. Will provide an update later. was (Author: ngrigor...@gmail.com): I did another round of testing and I can confirm my previous suspicion. If LCS goes into "STCS fallback" mode there seems to be some kind of "point of no return". After loading fairly large amount of data I end up with a number of large (from few Gb to 200+Gb) sstables. After that the cluster simply goes downhill - it never recovers. Even if there is no traffic except the repair service (DSE OpsCenter) the number of pending compactions never declines. It actually grows. Sstables also grow and grow in size until the moment one of the compactions runs out of disk space and crashes the node. Also I believe once in this state there is no way out. sstablesplit tool, as far as I understand, cannot be used with the live node. And the tool splits the data in single thread. I have measured its performance on my system, it processes about 13Mb/s on average, thus, to split all these large sstables it would take many DAYS. > LCS compaction low performance, many pending compactions, nodes are almost > idle > --- > > Key: CASSANDRA-7949 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7949 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: DSE 4.5.1-1, Cassandra 2.0.8 >Reporter: Nikolai Grigoriev > Attachments: iostats.txt, nodetool_compactionstats.txt, > nodetool_tpstats.txt, pending compactions 2day.png, system.log.gz, vmstat.txt > > > I've been evaluating new cluster of 15 nodes (32 core, 6x800Gb SSD disks + > 2x600Gb SAS, 128Gb RAM, OEL 6.5) and I've built a simulator that creates the > load similar to the load in our future product. Before running the simulator > I had to pre-generate enough data. This was done using Java code and DataStax > Java driver. To avoid going deep into details, two tables have been > generated. Each table currently has about 55M rows and between few dozens and > few thousands of columns in each row. > This data generation process was generating massive amount of non-overlapping > data. Thus, the activity was write-only and highly parallel. This is not the > type of the traffic that the system will have ultimately to deal with, it > will be mix of reads and updates to the existing data in the future. This is > just to explain the choice of LCS, not mentioning the expensive SSD disk > space. > At some point while generating the data I have noticed that the compactions > started to pile up. I knew that I was overloading the cluster but I still > wanted the genration test to complete. I was expecting to give the cluster > enough time to finish the pending compactions and get ready for real traffic. > However, after the storm of write requests have been stopped I have noticed > that the number of pending compactions remained constant (and even climbed up > a little bit) on all nodes. After trying to tune some parameters (like > setting the compaction bandwidth cap to 0) I have noticed a strange pattern: > the nodes were compacting one of the CFs in a single stream using virtu
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168841#comment-14168841 ] graham sanderson commented on CASSANDRA-7546: - Actually this is the first time I've looked at the Locks.java code in detail myself - it should probably not throw an AssertionError on failure (it should log) since it is optional - and maybe the methods should be renamed to indicate that it may be a noop > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Fix For: 2.1.1 > > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, > 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, > 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, > cassandra-2.1-7546-v2.txt, cassandra-2.1-7546-v3.txt, cassandra-2.1-7546.txt, > graph2_7546.png, graph3_7546.png, graph4_7546.png, graphs1.png, > hint_spikes.png, suggestion1.txt, suggestion1_21.txt, young_gen_gc.png > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] graham sanderson updated CASSANDRA-7546: Attachment: cassandra-2.1-7546-v3.txt Oops - my bad - Locks.java wasn't already committed (I though Benedict might have added it as part of something else), but it turns out I just had it in my working copy not commit... uploaded cassandra-2.1-7546-v3.txt which adds it back > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Fix For: 2.1.1 > > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, > 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, > 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, > cassandra-2.1-7546-v2.txt, cassandra-2.1-7546-v3.txt, cassandra-2.1-7546.txt, > graph2_7546.png, graph3_7546.png, graph4_7546.png, graphs1.png, > hint_spikes.png, suggestion1.txt, suggestion1_21.txt, young_gen_gc.png > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8106) Schema changes raises "comparators do not match or are not compatible" ConfigurationException
[ https://issues.apache.org/jira/browse/CASSANDRA-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168830#comment-14168830 ] Tommaso Barbugli commented on CASSANDRA-8106: - Couple of notes: while the response to the schema changes returns this error; changes take place. Since this problem appeared I always see that FlushWriter pool have > 0 values in the all time blocked column. > Schema changes raises "comparators do not match or are not compatible" > ConfigurationException > - > > Key: CASSANDRA-8106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8106 > Project: Cassandra > Issue Type: Bug >Reporter: Tommaso Barbugli > > I am running Cassandra 2.0.10 on 2 nodes; since the last few hours every > schema migration issued via CQL (both CREATE and ALTER tables) raises a > ConfigurationException "comparators do not match or are not compatible" > exception. > {code} > ERROR [Native-Transport-Requests:5802] 2014-10-12 22:48:12,237 > QueryMessage.java (line 131) Unexpected error during query > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.RuntimeException: > org.apache.cassandra.exceptions.ConfigurationException: comparators do not > match or are not compatible. > at > org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:413) > at > org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:285) > at > org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:223) > at > org.apache.cassandra.cql3.statements.CreateTableStatement.announceMigration(CreateTableStatement.java:121) > at > org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:79) > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:158) > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:175) > at > org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119) > at > org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:306) > at > org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) > at > org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > at > org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) > at > org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) > at > org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > Caused by: java.util.concurrent.ExecutionException: > java.lang.RuntimeException: > org.apache.cassandra.exceptions.ConfigurationException: comparators do not > match or are not compatible. > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:188) > at > org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:409) > ... 16 more > Caused by: java.lang.RuntimeException: > org.apache.cassandra.exceptions.ConfigurationException: comparators do not > match or are not compatible. > at org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1052) > at > org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:377) > at > org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:318) > at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:183) > at > org.apache.cassandra.service.MigrationManager$2.runMayThrow(MigrationManager.java:303) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > ... 3 more > Caused by: org.apache.cassandra.exceptions.ConfigurationException: > comparators do not match or are not compatible. > at > org.apache.cassandra.config.CFMetaData.validateCompatility(CFMetaData.java:1142) > at org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:1067) > at org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1048) > ... 10 more > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8106) Schema changes raises "comparators do not match or are not compatible" ConfigurationException
Tommaso Barbugli created CASSANDRA-8106: --- Summary: Schema changes raises "comparators do not match or are not compatible" ConfigurationException Key: CASSANDRA-8106 URL: https://issues.apache.org/jira/browse/CASSANDRA-8106 Project: Cassandra Issue Type: Bug Reporter: Tommaso Barbugli I am running Cassandra 2.0.10 on 2 nodes; since the last few hours every schema migration issued via CQL (both CREATE and ALTER tables) raises a ConfigurationException "comparators do not match or are not compatible" exception. {code} ERROR [Native-Transport-Requests:5802] 2014-10-12 22:48:12,237 QueryMessage.java (line 131) Unexpected error during query java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.cassandra.exceptions.ConfigurationException: comparators do not match or are not compatible. at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:413) at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:285) at org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:223) at org.apache.cassandra.cql3.statements.CreateTableStatement.announceMigration(CreateTableStatement.java:121) at org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:79) at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:158) at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:175) at org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119) at org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:306) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.cassandra.exceptions.ConfigurationException: comparators do not match or are not compatible. at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:409) ... 16 more Caused by: java.lang.RuntimeException: org.apache.cassandra.exceptions.ConfigurationException: comparators do not match or are not compatible. at org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1052) at org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:377) at org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:318) at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:183) at org.apache.cassandra.service.MigrationManager$2.runMayThrow(MigrationManager.java:303) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) ... 3 more Caused by: org.apache.cassandra.exceptions.ConfigurationException: comparators do not match or are not compatible. at org.apache.cassandra.config.CFMetaData.validateCompatility(CFMetaData.java:1142) at org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:1067) at org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1048) ... 10 more {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7949) LCS compaction low performance, many pending compactions, nodes are almost idle
[ https://issues.apache.org/jira/browse/CASSANDRA-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168822#comment-14168822 ] Nikolai Grigoriev commented on CASSANDRA-7949: -- I did another round of testing and I can confirm my previous suspicion. If LCS goes into "STCS fallback" mode there seems to be some kind of "point of no return". After loading fairly large amount of data I end up with a number of large (from few Gb to 200+Gb) sstables. After that the cluster simply goes downhill - it never recovers. Even if there is no traffic except the repair service (DSE OpsCenter) the number of pending compactions never declines. It actually grows. Sstables also grow and grow in size until the moment one of the compactions runs out of disk space and crashes the node. Also I believe once in this state there is no way out. sstablesplit tool, as far as I understand, cannot be used with the live node. And the tool splits the data in single thread. I have measured its performance on my system, it processes about 13Mb/s on average, thus, to split all these large sstables it would take many DAYS. > LCS compaction low performance, many pending compactions, nodes are almost > idle > --- > > Key: CASSANDRA-7949 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7949 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: DSE 4.5.1-1, Cassandra 2.0.8 >Reporter: Nikolai Grigoriev > Attachments: iostats.txt, nodetool_compactionstats.txt, > nodetool_tpstats.txt, pending compactions 2day.png, system.log.gz, vmstat.txt > > > I've been evaluating new cluster of 15 nodes (32 core, 6x800Gb SSD disks + > 2x600Gb SAS, 128Gb RAM, OEL 6.5) and I've built a simulator that creates the > load similar to the load in our future product. Before running the simulator > I had to pre-generate enough data. This was done using Java code and DataStax > Java driver. To avoid going deep into details, two tables have been > generated. Each table currently has about 55M rows and between few dozens and > few thousands of columns in each row. > This data generation process was generating massive amount of non-overlapping > data. Thus, the activity was write-only and highly parallel. This is not the > type of the traffic that the system will have ultimately to deal with, it > will be mix of reads and updates to the existing data in the future. This is > just to explain the choice of LCS, not mentioning the expensive SSD disk > space. > At some point while generating the data I have noticed that the compactions > started to pile up. I knew that I was overloading the cluster but I still > wanted the genration test to complete. I was expecting to give the cluster > enough time to finish the pending compactions and get ready for real traffic. > However, after the storm of write requests have been stopped I have noticed > that the number of pending compactions remained constant (and even climbed up > a little bit) on all nodes. After trying to tune some parameters (like > setting the compaction bandwidth cap to 0) I have noticed a strange pattern: > the nodes were compacting one of the CFs in a single stream using virtually > no CPU and no disk I/O. This process was taking hours. After that it would be > followed by a short burst of few dozens of compactions running in parallel > (CPU at 2000%, some disk I/O - up to 10-20%) and then getting stuck again for > many hours doing one compaction at time. So it looks like this: > # nodetool compactionstats > pending tasks: 3351 > compaction typekeyspace table completed > total unit progress >Compaction myks table_list1 66499295588 > 1910515889913 bytes 3.48% > Active compaction remaining time :n/a > # df -h > ... > /dev/sdb1.5T 637G 854G 43% /cassandra-data/disk1 > /dev/sdc1.5T 425G 1.1T 29% /cassandra-data/disk2 > /dev/sdd1.5T 429G 1.1T 29% /cassandra-data/disk3 > # find . -name **table_list1**Data** | grep -v snapshot | wc -l > 1310 > Among these files I see: > 1043 files of 161Mb (my sstable size is 160Mb) > 9 large files - 3 between 1 and 2Gb, 3 of 5-8Gb, 55Gb, 70Gb and 370Gb > 263 files of various sized - between few dozens of Kb and 160Mb > I've been running the heavy load for about 1,5days and it's been close to 3 > days after that and the number of pending compactions does not go down. > I have applied one of the not-so-obvious recommendations to disable > multithreaded compactions and that seems to be helping a bit - I see some > nodes started to have fewer pending compactions. About half of the cluster, > in fact. But even there I see they are sitting idle
[jira] [Comment Edited] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168561#comment-14168561 ] mck edited comment on CASSANDRA-6091 at 10/12/14 9:48 PM: -- I guess this^ approach falls apart once with increasing number of nodes in a cluster (the chances of adjacent splits with same dataNodes drops quickly), and it comes back to needing splits with multiple token ranges and CRR supporting that. But i still don't get why you *need* to have any thrift/CQL server-side change (at least to begin with)? For example this [patch|https://github.com/michaelsembwever/cassandra/pull/2/files] (note i'm intentionally waiting on feedback before tackling CFIF+CFRR). was (Author: michaelsembwever): I guess this^ approach falls apart once with increasing number of nodes in a cluster (the chances of adjacent splits with same dataNodes drops quickly), and it comes back to splits with multiple token ranges and CRR supporting that. But i still don't get why you *need* to have any thrift/CQL server-side change (at least to begin with)? > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5483) Repair tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168742#comment-14168742 ] Ben Chan commented on CASSANDRA-5483: - Okay, that covers all the questions that I had. TODO list: - Rebase to trunk (hopefully the last 3 months haven't made this too painful). - Miscellaneous remaining cruft from the 5483-16 cleanup. Possibly some code simplification (low-hanging fruit only). - Guard out-of-bounds-array access in TraceState#deserialize. The simplest thing I can think of is to have a "NONE" tracetype and return that for any unknown tracetype. - Fix format string "Sending completed merkle tree to %s for %s.%s" being traced verbatim. Barring anything unforseen, is that about it before it's good enough for a merge? It may be a few days before I actually get around to this. I had some computer trouble recently, so I'm going to have to set up my build environment from scratch. > Repair tracing > -- > > Key: CASSANDRA-5483 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5483 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Yuki Morishita >Assignee: Ben Chan >Priority: Minor > Labels: repair > Fix For: 3.0 > > Attachments: 5483-full-trunk.txt, > 5483-v06-04-Allow-tracing-ttl-to-be-configured.patch, > 5483-v06-05-Add-a-command-column-to-system_traces.events.patch, > 5483-v06-06-Fix-interruption-in-tracestate-propagation.patch, > 5483-v07-07-Better-constructor-parameters-for-DebuggableThreadPoolExecutor.patch, > 5483-v07-08-Fix-brace-style.patch, > 5483-v07-09-Add-trace-option-to-a-more-complete-set-of-repair-functions.patch, > 5483-v07-10-Correct-name-of-boolean-repairedAt-to-fullRepair.patch, > 5483-v08-11-Shorten-trace-messages.-Use-Tracing-begin.patch, > 5483-v08-12-Trace-streaming-in-Differencer-StreamingRepairTask.patch, > 5483-v08-13-sendNotification-of-local-traces-back-to-nodetool.patch, > 5483-v08-14-Poll-system_traces.events.patch, > 5483-v08-15-Limit-trace-notifications.-Add-exponential-backoff.patch, > 5483-v09-16-Fix-hang-caused-by-incorrect-exit-code.patch, > 5483-v10-17-minor-bugfixes-and-changes.patch, > 5483-v10-rebased-and-squashed-471f5cc.patch, 5483-v11-01-squashed.patch, > 5483-v11-squashed-nits.patch, 5483-v12-02-cassandra-yaml-ttl-doc.patch, > 5483-v13-608fb03-May-14-trace-formatting-changes.patch, > 5483-v14-01-squashed.patch, > 5483-v15-02-Hook-up-exponential-backoff-functionality.patch, > 5483-v15-03-Exact-doubling-for-exponential-backoff.patch, > 5483-v15-04-Re-add-old-StorageService-JMX-signatures.patch, > 5483-v15-05-Move-command-column-to-system_traces.sessions.patch, > 5483-v15.patch, ccm-repair-test, cqlsh-left-justify-text-columns.patch, > prerepair-vs-postbuggedrepair.diff, test-5483-system_traces-events.txt, > trunk@4620823-5483-v02-0001-Trace-filtering-and-tracestate-propagation.patch, > trunk@4620823-5483-v02-0002-Put-a-few-traces-parallel-to-the-repair-logging.patch, > tr...@8ebeee1-5483-v01-001-trace-filtering-and-tracestate-propagation.txt, > tr...@8ebeee1-5483-v01-002-simple-repair-tracing.txt, > v02p02-5483-v03-0003-Make-repair-tracing-controllable-via-nodetool.patch, > v02p02-5483-v04-0003-This-time-use-an-EnumSet-to-pass-boolean-repair-options.patch, > v02p02-5483-v05-0003-Use-long-instead-of-EnumSet-to-work-with-JMX.patch > > > I think it would be nice to log repair stats and results like query tracing > stores traces to system keyspace. With it, you don't have to lookup each log > file to see what was the status and how it performed the repair you invoked. > Instead, you can query the repair log with session ID to see the state and > stats of all nodes involved in that repair session. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8105) NPE for null embedded UDT inside set
[ https://issues.apache.org/jira/browse/CASSANDRA-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp reassigned CASSANDRA-8105: --- Assignee: Robert Stupp > NPE for null embedded UDT inside set > > > Key: CASSANDRA-8105 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8105 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Andy Ketskes >Assignee: Robert Stupp > Labels: cql > Fix For: 2.1.1 > > Attachments: 8105.txt > > > An NPE is thrown parsing an INSERT statement when a embedded UDT inside > another UDT is set to null inside a set. > This sounds very convoluted, but the examples below will hopefully make it > clear... > With the following definitions: > CREATE TYPE ut1 (a int, b int); > CREATE TYPE ut2 (j frozen, k int); > CREATE TYPE ut3 (i int, j frozen); > CREATE TABLE tab1 (x int PRIMARY KEY, y set>); > CREATE TABLE tab2 (x int PRIMARY KEY, y list>); > CREATE TABLE tab3 (x int PRIMARY KEY, y set>); > This query throws a NullPointerException: > INSERT INTO tab1 (x, y) VALUES (1, { { k: 1 } }); > These however doesn't: > INSERT INTO tab2 (x, y) VALUES (1, [ { k: 1 } ]); > INSERT INTO tab3 (x, y) VALUES (1, { { i: 1 } }); > So, the bug seems to be triggered only when the UDT is in a set, lists are > fine. If the embedded UDT is after the value specified in the query, the bug > doesn't seem to trigger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8105) NPE for null embedded UDT inside set
[ https://issues.apache.org/jira/browse/CASSANDRA-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-8105: Attachment: 8105.txt trivial fix for this > NPE for null embedded UDT inside set > > > Key: CASSANDRA-8105 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8105 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Andy Ketskes > Labels: cql > Fix For: 2.1.1 > > Attachments: 8105.txt > > > An NPE is thrown parsing an INSERT statement when a embedded UDT inside > another UDT is set to null inside a set. > This sounds very convoluted, but the examples below will hopefully make it > clear... > With the following definitions: > CREATE TYPE ut1 (a int, b int); > CREATE TYPE ut2 (j frozen, k int); > CREATE TYPE ut3 (i int, j frozen); > CREATE TABLE tab1 (x int PRIMARY KEY, y set>); > CREATE TABLE tab2 (x int PRIMARY KEY, y list>); > CREATE TABLE tab3 (x int PRIMARY KEY, y set>); > This query throws a NullPointerException: > INSERT INTO tab1 (x, y) VALUES (1, { { k: 1 } }); > These however doesn't: > INSERT INTO tab2 (x, y) VALUES (1, [ { k: 1 } ]); > INSERT INTO tab3 (x, y) VALUES (1, { { i: 1 } }); > So, the bug seems to be triggered only when the UDT is in a set, lists are > fine. If the embedded UDT is after the value specified in the query, the bug > doesn't seem to trigger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7813) Decide how to deal with conflict between native and user-defined functions
[ https://issues.apache.org/jira/browse/CASSANDRA-7813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7813: Attachment: 7813.txt > Decide how to deal with conflict between native and user-defined functions > -- > > Key: CASSANDRA-7813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7813 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Robert Stupp > Labels: cql > Fix For: 3.0 > > Attachments: 7813.txt > > > We have a bunch of native/hardcoded functions (now(), dateOf(), ...) and in > 3.0, user will be able to define new functions. Now, there is a very high > change that we will provide more native functions over-time (to be clear, I'm > not particularly for adding native functions for allthethings just because we > can, but it's clear that we should ultimately provide more than what we > have). Which begs the question: how do we want to deal with the problem of > adding a native function potentially breaking a previously defined > user-defined function? > A priori I see the following options (maybe there is more?): > # don't do anything specific, hoping that it won't happen often and consider > it a user problem if it does. > # reserve a big number of names that we're hoping will cover all future need. > # make native function and user-defined function syntactically distinct so it > cannot happen. > I'm not a huge fan of solution 1). Solution 2) is actually what we did for > UDT but I think it's somewhat less practical here: there is so much types > that it makes sense to provide natively and so it wasn't too hard to come up > with a reasonably small list of types name to reserve just in case. This > feels a lot harder for functions to me. > Which leaves solution 3). Since we already have the concept of namespaces for > functions, a simple idea would be to force user function to have namespace. > We could even allow that namespace to be empty as long as we force the > namespace separator (so we'd allow {{bar::foo}} and {{::foo}} for user > functions, but *not* {{foo}} which would be reserved for native function). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8105) NPE for null embedded UDT inside set
Andy Ketskes created CASSANDRA-8105: --- Summary: NPE for null embedded UDT inside set Key: CASSANDRA-8105 URL: https://issues.apache.org/jira/browse/CASSANDRA-8105 Project: Cassandra Issue Type: Bug Components: Core Reporter: Andy Ketskes An NPE is thrown parsing an INSERT statement when a embedded UDT inside another UDT is set to null inside a set. This sounds very convoluted, but the examples below will hopefully make it clear... With the following definitions: CREATE TYPE ut1 (a int, b int); CREATE TYPE ut2 (j frozen, k int); CREATE TYPE ut3 (i int, j frozen); CREATE TABLE tab1 (x int PRIMARY KEY, y set>); CREATE TABLE tab2 (x int PRIMARY KEY, y list>); CREATE TABLE tab3 (x int PRIMARY KEY, y set>); This query throws a NullPointerException: INSERT INTO tab1 (x, y) VALUES (1, { { k: 1 } }); These however doesn't: INSERT INTO tab2 (x, y) VALUES (1, [ { k: 1 } ]); INSERT INTO tab3 (x, y) VALUES (1, { { i: 1 } }); So, the bug seems to be triggered only when the UDT is in a set, lists are fine. If the embedded UDT is after the value specified in the query, the bug doesn't seem to trigger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7874) Validate functionality of different JSR-223 providers in UDFs
[ https://issues.apache.org/jira/browse/CASSANDRA-7874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7874: Attachment: 7874v3.txt New patch v3 (just rebased) > Validate functionality of different JSR-223 providers in UDFs > - > > Key: CASSANDRA-7874 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7874 > Project: Cassandra > Issue Type: Task > Components: Core >Reporter: Robert Stupp >Assignee: Robert Stupp > Labels: udf > Attachments: 7874.txt, 7874v2.txt, 7874v3.txt > > > CASSANDRA-7526 introduces ability to support optional JSR-223 providers like > Clojure, Jython, Groovy or JRuby. > This ticket is about to test functionality with these providers but not to > include them in C* distribution. > Expected result is a "how to" document, wiki page or similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7563) UserType, TupleType and collections in UDFs
[ https://issues.apache.org/jira/browse/CASSANDRA-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7563: Attachment: 7563.txt Patch {{7563.txt}} for user-types, tuple-types and collections. Requires CASSANDRA-7692 and CASSANDRA-8104 (this patch includes these) Functionality: * adds {{DataType}} for argument and return types to {{UDFunction}} * (de)serialization performed via {{DataType}} * no need to check for changed user types changed UTs (e.g. ALTER TYPE ADD) in UDFs - see {{UFTest.testJavaUserTypeMutation()}} > UserType, TupleType and collections in UDFs > --- > > Key: CASSANDRA-7563 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7563 > Project: Cassandra > Issue Type: Bug >Reporter: Robert Stupp >Assignee: Robert Stupp > Fix For: 3.0 > > Attachments: 7563-7740.txt, 7563.txt > > > * is Java Driver as a dependency required ? > * is it possible to extract parts of the Java Driver for UDT/TT/coll support ? > * CQL {{DROP TYPE}} must check UDFs > * must check keyspace access permissions (if those exist) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7563) UserType, TupleType and collections in UDFs
[ https://issues.apache.org/jira/browse/CASSANDRA-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168653#comment-14168653 ] Robert Stupp edited comment on CASSANDRA-7563 at 10/12/14 1:42 PM: --- Initial version for user-types, tuple-types, lists, sets, maps as argument & return types available in linked git branch. EDIT: working with Java + JavaScript UDFs was (Author: snazy): Initial version for user-types, tuple-types, lists, sets, maps as argument & return types available in linked git branch. > UserType, TupleType and collections in UDFs > --- > > Key: CASSANDRA-7563 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7563 > Project: Cassandra > Issue Type: Bug >Reporter: Robert Stupp >Assignee: Robert Stupp > Fix For: 3.0 > > Attachments: 7563-7740.txt > > > * is Java Driver as a dependency required ? > * is it possible to extract parts of the Java Driver for UDT/TT/coll support ? > * CQL {{DROP TYPE}} must check UDFs > * must check keyspace access permissions (if those exist) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8104) frozen broken / frozen collections in frozen tuple type
Robert Stupp created CASSANDRA-8104: --- Summary: frozen broken / frozen collections in frozen tuple type Key: CASSANDRA-8104 URL: https://issues.apache.org/jira/browse/CASSANDRA-8104 Project: Cassandra Issue Type: Bug Reporter: Robert Stupp Attachments: frozen-broken-patch.txt {{CREATE TABLE foo (key int primary key, tup frozen, set, map>>)}} Produces an NPE in {{CQL3Type.freeze()}}, because lists and sets have no keys. {{CREATE TABLE bar (key int primary key, tup frozen>, frozen>, frozen>>>)}} Produces some NPEs that prevent the correct error message being printed. Simple patch attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168561#comment-14168561 ] mck commented on CASSANDRA-6091: I guess this^ approach falls apart once with increasing number of nodes in a cluster (the chances of adjacent splits with same dataNodes drops quickly), and it comes back to splits with multiple token ranges and CRR supporting that. But i still don't get why you *need* to have any thrift/CQL server-side change (at least to begin with)? > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)