[
https://issues.apache.org/jira/browse/CASSANDRA-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030253#comment-13030253
]
Stu Hood commented on CASSANDRA-2618:
-------------------------------------
Are we sure this fixes the race? ArrayDeque isn't a concurrent datastructure,
so it's possible that its internals could race in such a way that it gets into
a bad state. The size field not matching the index, for example.
> DynamicSnitch race in adding latencies
> --------------------------------------
>
> Key: CASSANDRA-2618
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2618
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.0
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Priority: Minor
> Fix For: 0.7.6
>
> Attachments: 2618.txt
>
>
> ERROR 15:33:48,614 Fatal exception in thread Thread[ReadStage:264,5,main]
> java.lang.RuntimeException: java.util.NoSuchElementException
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.util.NoSuchElementException
> at
> java.util.concurrent.LinkedBlockingDeque.removeFirst(LinkedBlockingDeque.java:401)
> at
> java.util.concurrent.LinkedBlockingDeque.remove(LinkedBlockingDeque.java:621)
> at
> org.apache.cassandra.locator.AdaptiveLatencyTracker.add(DynamicEndpointSnitch.java:288)
> at
> org.apache.cassandra.locator.DynamicEndpointSnitch.receiveTiming(DynamicEndpointSnitch.java:202)
> at
> org.apache.cassandra.net.MessagingService.addLatency(MessagingService.java:152)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:642)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 15:33:48,615 Fatal exception in thread Thread[ReadStage:264,5,main]
> java.lang.RuntimeException: java.util.NoSuchElementException
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.util.NoSuchElementException
> at
> java.util.concurrent.LinkedBlockingDeque.removeFirst(LinkedBlockingDeque.java:401)
> at
> java.util.concurrent.LinkedBlockingDeque.remove(LinkedBlockingDeque.java:621)
> at
> org.apache.cassandra.locator.AdaptiveLatencyTracker.add(DynamicEndpointSnitch.java:288)
> at
> org.apache.cassandra.locator.DynamicEndpointSnitch.receiveTiming(DynamicEndpointSnitch.java:202)
> at
> org.apache.cassandra.net.MessagingService.addLatency(MessagingService.java:152)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:642)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> What is happening that AdaptiveLatencyTracker.add is trying to add a latency,
> but the deque is full, so it makes a second effort to remove an entry from
> the deque and then try to add again. However, when it tries to remove, the
> deque has already been emptied by DES.reset call clear() on all the ALTs.
> This bug has existed for a long time, but it's very rare and difficult to
> trigger.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
