[
https://issues.apache.org/jira/browse/CASSANDRA-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17800666#comment-17800666
]
Maxim Muzafarov commented on CASSANDRA-19082:
-
Hi [~kolargol] , this is a duplicate of CASSANDRA-14867 ?
> Histogram overflow causes client timeouts and message drops
> ---
>
> Key: CASSANDRA-19082
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19082
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zbyszek Z
>Priority: Normal
>
> Hi,
> We have recently noticed that sometimes this exception happens on our
> Cassandra cluster:
> {code:java}
> ERROR [ScheduledTasks:1] 2023-11-24 06:24:12,680 CassandraDaemon.java:244 -
> Exception in thread Thread[ScheduledTasks:1,5,main]
> java.lang.IllegalStateException: Unable to compute when histogram overflowed
> at
> org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoir$EstimatedHistogramReservoirSnapshot.getMean(DecayingEstimatedHistogramReservoir.java:472)
> at
> org.apache.cassandra.net.MessagingService.getDroppedMessagesLogs(MessagingService.java:1272)
> at
> org.apache.cassandra.net.MessagingService.logDroppedMessages(MessagingService.java:1244)
> at
> org.apache.cassandra.net.MessagingService.access$200(MessagingService.java:84)
> at
> org.apache.cassandra.net.MessagingService$4.run(MessagingService.java:512)
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
> at java.lang.Thread.run(Thread.java:750)
> {code}
> It happens on all 6 nodes at the same time. Also we see increased client
> timeouts and dropped READ and READ_RESPONSE messages. Our Cassandra is
> 3.11.16, 2 DC setup, 6 node in each DC. RF is 3. I have searched issues but
> could not find exactly same issue causing messages to be dropped. Any
> suggestion would be appreciated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
