[ https://issues.apache.org/jira/browse/CASSANDRA-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492245#comment-13492245 ]
Mck SembWever edited comment on CASSANDRA-4417 at 11/7/12 10:20 AM: -------------------------------------------------------------------- Sylvain, here's log from one node. For most of the log we were running 1.0.8. And then at line 2883399 we upgraded (and this was the first node to upgrade) to 1.1.6. The error msg comes every few seconds. Our counters are sub-columns inside supercolumns. We completed the upgrade on all nodes. Then restarted again (because jna was missing). We are now running upgradesstables but that's not in this logfile. The error msgs still appear. An operational problem we've had recently is that we had one node down for ~one month (faulty raid controller) and when we finally brought the node back into the cluster nightly repairs would never finish. In the end we just disabled nightly repairs (we don't have tombstones) with the plan that an upgrade and upgradesstables would bring us back to a state where repairs would work again. I have no idea if this can be related. was (Author: michaelsembwever): Sylvain, here's log from one node. For most of the log we were running 1.0.8. And then at line 2883399 we upgraded (and this was the first node to upgrade) to 1.1.6. The error msg comes every few seconds. Our counters are sub-columns inside supercolumns. We completed the upgrade on all nodes. Then restarted again (because jna was missing). We are now running upgradesstables but that's not in this logfile. The error msgs still appear. An operational problem we're had recently is that we had one node down for ~one month (faulty raid controller) and when we finally brought the node back into the cluster nightly repairs would never finish. In the end we just disabled nightly repairs (we don't have tombstones) with the plan that an upgrade and upgradesstables would bring us back to a state where repairs would work again. I have no idea if this can be related. > invalid counter shard detected > ------------------------------- > > Key: CASSANDRA-4417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4417 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.1.1 > Environment: Amazon Linux > Reporter: Senthilvel Rangaswamy > Attachments: cassandra-mck.log.bz2, err.txt > > > Seeing errors like these: > 2012-07-06_07:00:27.22662 ERROR 07:00:27,226 invalid counter shard detected; > (17bfd850-ac52-11e1-0000-6ecd0b5b61e7, 1, 13) and > (17bfd850-ac52-11e1-0000-6ecd0b5b61e7, 1, 1) differ only in count; will pick > highest to self-heal; this indicates a bug or corruption generated a bad > counter shard > What does it mean ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira