[
https://issues.apache.org/jira/browse/CASSANDRA-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13951109#comment-13951109
]
Benedict commented on CASSANDRA-6948:
-------------------------------------
bq. Any chance this is somehow related to the issue where I'm not seeing DDLs
apply across all nodes even though they're shown as successful?
It's quite possible, yes, although cannot say for certain. If you don't see any
further DDL problems it's probably down to this, but otherwise please file a
separate issue.
> Native & ReplicateOnWriteStage AssertionError: Callback already exists
> ----------------------------------------------------------------------
>
> Key: CASSANDRA-6948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6948
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Keith Wright
> Assignee: Benedict
> Attachments: 6948.debug.txt, Screen Shot 2014-03-28 at 11.27.56
> AM.png, Screen Shot 2014-03-28 at 11.29.24 AM.png, cassandra.log.min,
> cassandra.yaml, logs.old.tar.gz, logs.tar.gz, system.log.1.gz, system.log.gz
>
>
> We are using the datastax driver and have seen the following exceptions
> appear periodically in our cluster in "groups". The client will gradually
> start sending less write requests to the nodes where this occurs the most
> often. This may be related to CASSANDRA-6476. Exception from logs:
> ERROR [Native-Transport-Requests:7636] 2014-03-28 13:32:10,638
> ErrorMessage.java (line 222) Unexpected exception during request
> java.lang.AssertionError: Callback already exists for id -1665979622!
> (CallbackInfo(target=/10.106.160.84,
> callback=org.apache.cassandra.service.WriteResponseHandler@5d36d8ea,
> serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
> at
> org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
> at
> org.apache.cassandra.service.StorageProxy.mutateCounter(StorageProxy.java:984)
> at
> org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:449)
> at
> org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:524)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:521)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:505)
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
> at
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:358)
> at
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:131)
> at
> org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:304)
> at
> org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
> at
> org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> ERROR [ReplicateOnWriteStage:102766] 2014-03-28 13:32:10,638
> CassandraDaemon.java (line 196) Exception in thread
> Thread[ReplicateOnWriteStage:102766,5,main]
> java.lang.AssertionError: Callback already exists for id -1665979620!
> (CallbackInfo(target=/10.106.160.84,
> callback=org.apache.cassandra.service.WriteResponseHandler@3bdb1a75,
> serializer=org.apache.cassandra.db.WriteResponse$WriteResponseSerializer@6ed37f0b))
> at
> org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:549)
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:601)
> at
> org.apache.cassandra.service.StorageProxy.sendToHintedEndpoints(StorageProxy.java:806)
> at
> org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1074)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1896)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
