I also encounter the schema disagreement in my 0.8.1 cluster today… The disagreement occurs when I create a column family using the hector api, and I found the following errors in my cassandra/system.log
ERROR [pool-2-thread-99] 2011-08-03 11:21:18,051 Cassandra.java (line 3378) Internal error processing remove java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:73) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337) at org.apache.cassandra.service.StorageProxy.insertLocal(StorageProxy.java:360) at org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:241) at org.apache.cassandra.service.StorageProxy.access$000(StorageProxy.java:62) at org.apache.cassandra.service.StorageProxy$1.apply(StorageProxy.java:99) at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:210) at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:154) at org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:560) at org.apache.cassandra.thrift.CassandraServer.internal_remove(CassandraServer.java:539) at org.apache.cassandra.thrift.CassandraServer.remove(CassandraServer.java:547) at org.apache.cassandra.thrift.Cassandra$Processor$remove.process(Cassandra.java:3370) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) And when I try to decommission, I got this: ERROR [pool-2-thread-90] 2011-08-03 11:24:35,611 Cassandra.java (line 3462) Internal error processing batch_mutate java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:73) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337) at org.apache.cassandra.service.StorageProxy.insertLocal(StorageProxy.java:360) at org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:241) at org.apache.cassandra.service.StorageProxy.access$000(StorageProxy.java:62) at org.apache.cassandra.service.StorageProxy$1.apply(StorageProxy.java:99) at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:210) at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:154) at org.apache.cassandra.thrift.CassandraServer.doInsert(CassandraServer.java:560) at org.apache.cassandra.thrift.CassandraServer.internal_batch_mutate(CassandraServer.java:511) at org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:519) at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.process(Cassandra.java:3454) at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) What does this mean? Thanks. -- Dikang Gu 0086 - 18611140205 On Tuesday, August 2, 2011 at 6:04 PM, aaron morton wrote: > Hang on, using brain now. > > That is triggering a small bug in the code see > https://issues.apache.org/jira/browse/CASSANDRA-2984 > > For not just remove the column meta data. > > Cheers > > ----------------- > Aaron Morton > Freelance Cassandra Developer > @aaronmorton > http://www.thelastpickle.com > > > > > > On 2 Aug 2011, at 21:19, aaron morton wrote: > > What do you see when you run describe cluster; in the cassandra-cli ? Whats > > the exact error you get and is there anything in the server side logs ? > > > > Have you added other CF's before adding this one ? Did the schema agree > > before starting this statement? > > > > I ran the statement below on the current trunk and it worked. > > > > Cheers > > > > ----------------- > > Aaron Morton > > Freelance Cassandra Developer > > @aaronmorton > > http://www.thelastpickle.com > > > > > > > > > > > > On 2 Aug 2011, at 12:08, Dikang Gu wrote: > > > I thought the schema disagree problem was already solved in 0.8.1... > > > > > > On possible solution is to decommission the disagree node and rejoin it. > > > > > > > > > On Tue, Aug 2, 2011 at 8:01 AM, Yi Yang <yy...@me.com> wrote: > > > > Dear all, > > > > > > > > I'm always meeting mp with schema disagree problems while trying to > > > > create a column family like this, using cassandra-cli: > > > > > > > > create column family sd > > > > with column_type = 'Super' > > > > and key_validation_class = 'UUIDType' > > > > and comparator = 'LongType' > > > > and subcomparator = 'UTF8Type' > > > > and column_metadata = [ > > > > { > > > > column_name: 'time', > > > > validation_class : 'LongType' > > > > },{ > > > > column_name: 'open', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'high', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'low', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'close', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'volumn', > > > > validation_class : 'LongType' > > > > },{ > > > > column_name: 'splitopen', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'splithigh', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'splitlow', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'splitclose', > > > > validation_class : 'FloatType' > > > > },{ > > > > column_name: 'splitvolume', > > > > validation_class : 'LongType' > > > > },{ > > > > column_name: 'splitclose', > > > > validation_class : 'FloatType' > > > > } > > > > ] > > > > ; > > > > > > > > I've tried to erase everything and restart Cassandra but this still > > > > happens. But when I clear the column_metadata section this no more > > > > disagreement error. Do you have any idea why this happens? > > > > > > > > Environment: 2 VMs, using the same harddrive, Cassandra 0.8.1, Ubuntu > > > > 10.04 > > > > This is for testing only. We'll move to dedicated servers later. > > > > > > > > Best regards, > > > > Yi > > > > > > > > > > > > > -- > > > Dikang Gu > > > > > > 0086 - 18611140205 > > > > >
