[
https://issues.apache.org/jira/browse/CASSANDRA-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203700#comment-13203700
]
Sylvain Lebresne commented on CASSANDRA-3819:
---------------------------------------------
I'm not able to reproduce this, so any info on how to reproduce, or someone did
reproduce, the commit logs that bock the restart would help. The attached
stack-trace attached unfortunately doesn't help much, except that something
that was read from the commit log was wrong.
> Cannot restart server after making schema changes to composite CFs
> ------------------------------------------------------------------
>
> Key: CASSANDRA-3819
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3819
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.6, 1.0.7
> Environment: Ubuntu 11.0.4
> Reporter: Huy Le
> Assignee: Sylvain Lebresne
> Fix For: 1.0.8
>
>
> This JIRA is for issue discussed in this thread
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7150978.html.
> We were using version 1.0.6. We added new keyspace using built-in composite
> data type. We then decided to change the schema, specifically just the CF
> names, so we dropped the keyspace. We recreated the key space with different
> CF names in the key space.
> There were a lot of uncommitted data in commit logs. Data dated back before
> the original key space was created. When we restarted the server, the server
> failed when it read it the commit logs, and the server stopped. Here is
> snippet of the stack trace:
> {code}
> -3881-11e1-ac7f-12313d23ead3:true:4@1326223353559001,])}
> DEBUG 18:02:01,057 Reading mutation at 66336992
> DEBUG 18:02:01,058 replaying mutation for
> Springpad.696d6167652d7363616c65722d6d657461:
> {ColumnFamily(CassandraOrderedQueue
> [0,eb321490-3881-11e1-ac7f-12313d23ead3:true:4@1326223353560000,])}
> DEBUG 18:02:01,058 Reading mutation at 66337118
> DEBUG 18:02:01,058 replaying mutation for
> Springpad.737072696e674d6f64656c44617461626173652d6d657461:
> {ColumnFamily(CassandraOrderedQueue
> [0,80dc0cd0-3bc0-11e1-83a8-12313d23ead3:false:8@1326223386668000,])}
> DEBUG 18:02:01,058 Reading mutation at 66337255
> DEBUG 18:02:01,058 replaying mutation for
> system.38363233616337302d336263302d313165312d303030302d323366623834646463346633:
> {ColumnFamily(Schema
> [Avro/Schema:false:2725@1326223386807,Backups:false:431@1326223386807,Springpad:false:10814@1326223386807,SpringpadGraph:false:2931@1326223386807,])}
> DEBUG 18:02:01,059 Reading mutation at 66354352
> DEBUG 18:02:01,059 replaying mutation for
> system.4d6967726174696f6e73204b6579: {ColumnFamily(Migrations
> [8623ac70-3bc0-11e1-0000-23fb84ddc4f3:false:23728@1326223386812,])}
> DEBUG 18:02:01,059 Reading mutation at 66378184
> DEBUG 18:02:01,059 replaying mutation for
> system.4c617374204d6967726174696f6e: {ColumnFamily(Schema [Last
> Migration:false:16@1326223386812,])}
> DEBUG 18:02:01,059 Reading mutation at 66378302
> INFO 18:02:01,060 Finished reading
> /mnt/cassandra/commitlog/CommitLog-1325861435420.log
> ERROR 18:02:01,061 Exception encountered during startup
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:247)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:57)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:66)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:129)
> at org.apache.cassandra.db.Column.getString(Column.java:250)
> at
> org.apache.cassandra.db.marshal.AbstractType.getColumnsString(AbstractType.java:137)
> at
> org.apache.cassandra.db.ColumnFamily.toString(ColumnFamily.java:280)
> at org.apache.commons.lang.ObjectUtils.toString(ObjectUtils.java:241)
> at org.apache.commons.lang.StringUtils.join(StringUtils.java:3073)
> at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:301)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:172)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:215)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:356)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:107)
> Exception encountered during startup: null
> {code}
> Sample original CF schema:
> {code}
> create column family InEdges
> with column_type = 'Standard'
> and comparator =
> 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> and default_validation_class = 'BytesType'
> and key_validation_class = 'UTF8Type'
> and rows_cached = 0.0
> and row_cache_save_period = 0
> and row_cache_keys_to_save = 0
> and keys_cached = 0.0
> and key_cache_save_period = 0
> and read_repair_chance = 0.0
> and gc_grace = 0
> and min_compaction_threshold = 4
> and max_compaction_threshold = 32
> and replicate_on_write = true
> and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
> and compaction_strategy =
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
> {code}
> And the updated schema (just CF name change):
> {code}
> create column family NewInEdges
> with column_type = 'Standard'
> and comparator =
> 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> and default_validation_class = 'BytesType'
> and key_validation_class = 'UTF8Type'
> and rows_cached = 0.0
> and row_cache_save_period = 0
> and row_cache_keys_to_save = 0
> and keys_cached = 0.0
> and key_cache_save_period = 0
> and read_repair_chance = 0.0
> and gc_grace = 0
> and min_compaction_threshold = 4
> and max_compaction_threshold = 32
> and replicate_on_write = true
> and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
> and compaction_strategy =
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
> {code}
> This issue happens every time composite type is used and workaround can be
> found at
> (http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7226863.html).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
