[
https://issues.apache.org/jira/browse/CASSANDRA-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200475#comment-13200475
]
Huy Le commented on CASSANDRA-3819:
-----------------------------------
I did not try to reproduce it. Another user mentioned in this thread
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7226863.html
that it happened every time.
As for me, I have 3 environments. This issue happened on an environment where
there where uncommitted logs accumulated before the schema changes. Two other
environments have committed logs fully flushed before the restart did not
suffer this issue. In my environments, there were data inserted into the node
before and after schema changes.
> Cannot restart server after making schema changes to composite CFs
> ------------------------------------------------------------------
>
> Key: CASSANDRA-3819
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3819
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.6, 1.0.7
> Environment: Ubuntu 11.0.4
> Reporter: Huy Le
> Assignee: Sylvain Lebresne
> Fix For: 1.0.8
>
>
> This JIRA is for issue discussed in this thread
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7150978.html.
> We were using version 1.0.6. We added new keyspace using built-in composite
> data type. We then decided to change the schema, specifically just the CF
> names, so we dropped the keyspace. We recreated the key space with different
> CF names in the key space.
> There were a lot of uncommitted data in commit logs. Data dated back before
> the original key space was created. When we restarted the server, the server
> failed when it read it the commit logs, and the server stopped. Here is
> snippet of the stack trace:
> {code}
> -3881-11e1-ac7f-12313d23ead3:true:4@1326223353559001,])}
> DEBUG 18:02:01,057 Reading mutation at 66336992
> DEBUG 18:02:01,058 replaying mutation for
> Springpad.696d6167652d7363616c65722d6d657461:
> {ColumnFamily(CassandraOrderedQueue
> [0,eb321490-3881-11e1-ac7f-12313d23ead3:true:4@1326223353560000,])}
> DEBUG 18:02:01,058 Reading mutation at 66337118
> DEBUG 18:02:01,058 replaying mutation for
> Springpad.737072696e674d6f64656c44617461626173652d6d657461:
> {ColumnFamily(CassandraOrderedQueue
> [0,80dc0cd0-3bc0-11e1-83a8-12313d23ead3:false:8@1326223386668000,])}
> DEBUG 18:02:01,058 Reading mutation at 66337255
> DEBUG 18:02:01,058 replaying mutation for
> system.38363233616337302d336263302d313165312d303030302d323366623834646463346633:
> {ColumnFamily(Schema
> [Avro/Schema:false:2725@1326223386807,Backups:false:431@1326223386807,Springpad:false:10814@1326223386807,SpringpadGraph:false:2931@1326223386807,])}
> DEBUG 18:02:01,059 Reading mutation at 66354352
> DEBUG 18:02:01,059 replaying mutation for
> system.4d6967726174696f6e73204b6579: {ColumnFamily(Migrations
> [8623ac70-3bc0-11e1-0000-23fb84ddc4f3:false:23728@1326223386812,])}
> DEBUG 18:02:01,059 Reading mutation at 66378184
> DEBUG 18:02:01,059 replaying mutation for
> system.4c617374204d6967726174696f6e: {ColumnFamily(Schema [Last
> Migration:false:16@1326223386812,])}
> DEBUG 18:02:01,059 Reading mutation at 66378302
> INFO 18:02:01,060 Finished reading
> /mnt/cassandra/commitlog/CommitLog-1325861435420.log
> ERROR 18:02:01,061 Exception encountered during startup
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:247)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:57)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:66)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:129)
> at org.apache.cassandra.db.Column.getString(Column.java:250)
> at
> org.apache.cassandra.db.marshal.AbstractType.getColumnsString(AbstractType.java:137)
> at
> org.apache.cassandra.db.ColumnFamily.toString(ColumnFamily.java:280)
> at org.apache.commons.lang.ObjectUtils.toString(ObjectUtils.java:241)
> at org.apache.commons.lang.StringUtils.join(StringUtils.java:3073)
> at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:301)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:172)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:215)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:356)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:107)
> Exception encountered during startup: null
> {code}
> Sample original CF schema:
> {code}
> create column family InEdges
> with column_type = 'Standard'
> and comparator =
> 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> and default_validation_class = 'BytesType'
> and key_validation_class = 'UTF8Type'
> and rows_cached = 0.0
> and row_cache_save_period = 0
> and row_cache_keys_to_save = 0
> and keys_cached = 0.0
> and key_cache_save_period = 0
> and read_repair_chance = 0.0
> and gc_grace = 0
> and min_compaction_threshold = 4
> and max_compaction_threshold = 32
> and replicate_on_write = true
> and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
> and compaction_strategy =
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
> {code}
> And the updated schema (just CF name change):
> {code}
> create column family NewInEdges
> with column_type = 'Standard'
> and comparator =
> 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
> and default_validation_class = 'BytesType'
> and key_validation_class = 'UTF8Type'
> and rows_cached = 0.0
> and row_cache_save_period = 0
> and row_cache_keys_to_save = 0
> and keys_cached = 0.0
> and key_cache_save_period = 0
> and read_repair_chance = 0.0
> and gc_grace = 0
> and min_compaction_threshold = 4
> and max_compaction_threshold = 32
> and replicate_on_write = true
> and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
> and compaction_strategy =
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
> {code}
> This issue happens every time composite type is used and workaround can be
> found at
> (http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7226863.html).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
