[jira] [Commented] (CASSANDRA-4134) Do not send hints before a node is fully up
[ https://issues.apache.org/jira/browse/CASSANDRA-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251141#comment-13251141 ] Jonathan Ellis commented on CASSANDRA-4134: --- Hmm, yes (on the sending side). > Do not send hints before a node is fully up > --- > > Key: CASSANDRA-4134 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4134 > Project: Cassandra > Issue Type: Bug >Reporter: Joaquin Casares >Priority: Minor > > After seeing this on a cluster and working with Pavel, we have seen the > following errors disappear after all migrations have been applied: > {noformat} > ERROR [MutationStage:1] 2012-04-09 18:16:00,240 RowMutationVerbHandler.java > (line 61) Error in row mutation > org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find > cfId=1028 > at > org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:129) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:401) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:409) > at org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:357) > at > org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:42) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > and > ERROR [ReadStage:69] 2012-04-09 18:16:01,715 AbstractCassandraDaemon.java > (line 139) Fatal exception in thread Thread[ReadStage:69,5,main] > java.lang.IllegalArgumentException: Unknown ColumnFamily content_indexes in > keyspace linkcurrent > at org.apache.cassandra.config.Schema.getComparator(Schema.java:223) > at > org.apache.cassandra.db.ColumnFamily.getComparatorFor(ColumnFamily.java:300) > at > org.apache.cassandra.db.ReadCommand.getComparator(ReadCommand.java:92) > at > org.apache.cassandra.db.SliceByNamesReadCommand.(SliceByNamesReadCommand.java:44) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:106) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:74) > at > org.apache.cassandra.db.ReadCommandSerializer.deserialize(ReadCommand.java:132) > at > org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:51) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > {noformat} > It seems as though as soon as the correct Migration is applied, the Hints are > accepted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4134) Do not send hints before a node is fully up
[ https://issues.apache.org/jira/browse/CASSANDRA-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251138#comment-13251138 ] Brandon Williams commented on CASSANDRA-4134: - Isn't this exactly what HHOM.waitForSchemaAgreement is doing though? > Do not send hints before a node is fully up > --- > > Key: CASSANDRA-4134 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4134 > Project: Cassandra > Issue Type: Bug >Reporter: Joaquin Casares >Priority: Minor > > After seeing this on a cluster and working with Pavel, we have seen the > following errors disappear after all migrations have been applied: > {noformat} > ERROR [MutationStage:1] 2012-04-09 18:16:00,240 RowMutationVerbHandler.java > (line 61) Error in row mutation > org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find > cfId=1028 > at > org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:129) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:401) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:409) > at org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:357) > at > org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:42) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > and > ERROR [ReadStage:69] 2012-04-09 18:16:01,715 AbstractCassandraDaemon.java > (line 139) Fatal exception in thread Thread[ReadStage:69,5,main] > java.lang.IllegalArgumentException: Unknown ColumnFamily content_indexes in > keyspace linkcurrent > at org.apache.cassandra.config.Schema.getComparator(Schema.java:223) > at > org.apache.cassandra.db.ColumnFamily.getComparatorFor(ColumnFamily.java:300) > at > org.apache.cassandra.db.ReadCommand.getComparator(ReadCommand.java:92) > at > org.apache.cassandra.db.SliceByNamesReadCommand.(SliceByNamesReadCommand.java:44) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:106) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:74) > at > org.apache.cassandra.db.ReadCommandSerializer.deserialize(ReadCommand.java:132) > at > org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:51) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > {noformat} > It seems as though as soon as the correct Migration is applied, the Hints are > accepted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4134) Do not send hints before a node is fully up
[ https://issues.apache.org/jira/browse/CASSANDRA-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251108#comment-13251108 ] Jonathan Ellis commented on CASSANDRA-4134: --- But it *is* up, it just doesn't have the most recent schema (which it has no way of knowing). > Do not send hints before a node is fully up > --- > > Key: CASSANDRA-4134 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4134 > Project: Cassandra > Issue Type: Bug >Reporter: Joaquin Casares >Priority: Minor > > After seeing this on a cluster and working with Pavel, we have seen the > following errors disappear after all migrations have been applied: > {noformat} > ERROR [MutationStage:1] 2012-04-09 18:16:00,240 RowMutationVerbHandler.java > (line 61) Error in row mutation > org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find > cfId=1028 > at > org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:129) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:401) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:409) > at org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:357) > at > org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:42) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > and > ERROR [ReadStage:69] 2012-04-09 18:16:01,715 AbstractCassandraDaemon.java > (line 139) Fatal exception in thread Thread[ReadStage:69,5,main] > java.lang.IllegalArgumentException: Unknown ColumnFamily content_indexes in > keyspace linkcurrent > at org.apache.cassandra.config.Schema.getComparator(Schema.java:223) > at > org.apache.cassandra.db.ColumnFamily.getComparatorFor(ColumnFamily.java:300) > at > org.apache.cassandra.db.ReadCommand.getComparator(ReadCommand.java:92) > at > org.apache.cassandra.db.SliceByNamesReadCommand.(SliceByNamesReadCommand.java:44) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:106) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:74) > at > org.apache.cassandra.db.ReadCommandSerializer.deserialize(ReadCommand.java:132) > at > org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:51) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > {noformat} > It seems as though as soon as the correct Migration is applied, the Hints are > accepted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4134) Do not send hints before a node is fully up
[ https://issues.apache.org/jira/browse/CASSANDRA-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251103#comment-13251103 ] Joaquin Casares commented on CASSANDRA-4134: When talking with Pavel it seemed more like: We don't send nodes that aren't up hints. Or perhaps, nodes shouldn't accept hints until they are up. > Do not send hints before a node is fully up > --- > > Key: CASSANDRA-4134 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4134 > Project: Cassandra > Issue Type: Bug >Reporter: Joaquin Casares >Priority: Minor > > After seeing this on a cluster and working with Pavel, we have seen the > following errors disappear after all migrations have been applied: > {noformat} > ERROR [MutationStage:1] 2012-04-09 18:16:00,240 RowMutationVerbHandler.java > (line 61) Error in row mutation > org.apache.cassandra.db.UnserializableColumnFamilyException: Couldn't find > cfId=1028 > at > org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:129) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:401) > at > org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:409) > at org.apache.cassandra.db.RowMutation.fromBytes(RowMutation.java:357) > at > org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:42) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > and > ERROR [ReadStage:69] 2012-04-09 18:16:01,715 AbstractCassandraDaemon.java > (line 139) Fatal exception in thread Thread[ReadStage:69,5,main] > java.lang.IllegalArgumentException: Unknown ColumnFamily content_indexes in > keyspace linkcurrent > at org.apache.cassandra.config.Schema.getComparator(Schema.java:223) > at > org.apache.cassandra.db.ColumnFamily.getComparatorFor(ColumnFamily.java:300) > at > org.apache.cassandra.db.ReadCommand.getComparator(ReadCommand.java:92) > at > org.apache.cassandra.db.SliceByNamesReadCommand.(SliceByNamesReadCommand.java:44) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:106) > at > org.apache.cassandra.db.SliceByNamesReadCommandSerializer.deserialize(SliceByNamesReadCommand.java:74) > at > org.apache.cassandra.db.ReadCommandSerializer.deserialize(ReadCommand.java:132) > at > org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:51) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > {noformat} > It seems as though as soon as the correct Migration is applied, the Hints are > accepted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira