[ https://issues.apache.org/jira/browse/CASSANDRA-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-3819. --------------------------------------- Resolution: Cannot Reproduce Fix Version/s: (was: 1.0.10) Assignee: (was: Sylvain Lebresne) Resolving as cantrepro. Carlo/Huy, if you or anyone else can give us a test case to work on, please re-open. > Cannot restart server after making schema changes to composite CFs > ------------------------------------------------------------------ > > Key: CASSANDRA-3819 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3819 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.0.6, 1.0.7 > Environment: Ubuntu 11.0.4 > Reporter: Huy Le > > This JIRA is for issue discussed in this thread > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7150978.html. > We were using version 1.0.6. We added new keyspace using built-in composite > data type. We then decided to change the schema, specifically just the CF > names, so we dropped the keyspace. We recreated the key space with different > CF names in the key space. > There were a lot of uncommitted data in commit logs. Data dated back before > the original key space was created. When we restarted the server, the server > failed when it read it the commit logs, and the server stopped. Here is > snippet of the stack trace: > {code} > -3881-11e1-ac7f-12313d23ead3:true:4@1326223353559001,])} > DEBUG 18:02:01,057 Reading mutation at 66336992 > DEBUG 18:02:01,058 replaying mutation for > Springpad.696d6167652d7363616c65722d6d657461: > {ColumnFamily(CassandraOrderedQueue > [0,eb321490-3881-11e1-ac7f-12313d23ead3:true:4@1326223353560000,])} > DEBUG 18:02:01,058 Reading mutation at 66337118 > DEBUG 18:02:01,058 replaying mutation for > Springpad.737072696e674d6f64656c44617461626173652d6d657461: > {ColumnFamily(CassandraOrderedQueue > [0,80dc0cd0-3bc0-11e1-83a8-12313d23ead3:false:8@1326223386668000,])} > DEBUG 18:02:01,058 Reading mutation at 66337255 > DEBUG 18:02:01,058 replaying mutation for > system.38363233616337302d336263302d313165312d303030302d323366623834646463346633: > {ColumnFamily(Schema > [Avro/Schema:false:2725@1326223386807,Backups:false:431@1326223386807,Springpad:false:10814@1326223386807,SpringpadGraph:false:2931@1326223386807,])} > DEBUG 18:02:01,059 Reading mutation at 66354352 > DEBUG 18:02:01,059 replaying mutation for > system.4d6967726174696f6e73204b6579: {ColumnFamily(Migrations > [8623ac70-3bc0-11e1-0000-23fb84ddc4f3:false:23728@1326223386812,])} > DEBUG 18:02:01,059 Reading mutation at 66378184 > DEBUG 18:02:01,059 replaying mutation for > system.4c617374204d6967726174696f6e: {ColumnFamily(Schema [Last > Migration:false:16@1326223386812,])} > DEBUG 18:02:01,059 Reading mutation at 66378302 > INFO 18:02:01,060 Finished reading > /mnt/cassandra/commitlog/CommitLog-1325861435420.log > ERROR 18:02:01,061 Exception encountered during startup > java.lang.IllegalArgumentException > at java.nio.Buffer.limit(Buffer.java:247) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:57) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:66) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:129) > at org.apache.cassandra.db.Column.getString(Column.java:250) > at > org.apache.cassandra.db.marshal.AbstractType.getColumnsString(AbstractType.java:137) > at > org.apache.cassandra.db.ColumnFamily.toString(ColumnFamily.java:280) > at org.apache.commons.lang.ObjectUtils.toString(ObjectUtils.java:241) > at org.apache.commons.lang.StringUtils.join(StringUtils.java:3073) > at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133) > at > org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:301) > at > org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:172) > at > org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:215) > at > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:356) > at > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:107) > Exception encountered during startup: null > {code} > Sample original CF schema: > {code} > create column family InEdges > with column_type = 'Standard' > and comparator = > 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)' > and default_validation_class = 'BytesType' > and key_validation_class = 'UTF8Type' > and rows_cached = 0.0 > and row_cache_save_period = 0 > and row_cache_keys_to_save = 0 > and keys_cached = 0.0 > and key_cache_save_period = 0 > and read_repair_chance = 0.0 > and gc_grace = 0 > and min_compaction_threshold = 4 > and max_compaction_threshold = 32 > and replicate_on_write = true > and row_cache_provider = 'ConcurrentLinkedHashCacheProvider' > and compaction_strategy = > 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'; > {code} > And the updated schema (just CF name change): > {code} > create column family NewInEdges > with column_type = 'Standard' > and comparator = > 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)' > and default_validation_class = 'BytesType' > and key_validation_class = 'UTF8Type' > and rows_cached = 0.0 > and row_cache_save_period = 0 > and row_cache_keys_to_save = 0 > and keys_cached = 0.0 > and key_cache_save_period = 0 > and read_repair_chance = 0.0 > and gc_grace = 0 > and min_compaction_threshold = 4 > and max_compaction_threshold = 32 > and replicate_on_write = true > and row_cache_provider = 'ConcurrentLinkedHashCacheProvider' > and compaction_strategy = > 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'; > {code} > This issue happens every time composite type is used and workaround can be > found at > (http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7226863.html). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira