[ https://issues.apache.org/jira/browse/CASSANDRA-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800666#comment-17800666 ]
Maxim Muzafarov commented on CASSANDRA-19082: --------------------------------------------- Hi [~kolargol] , this is a duplicate of CASSANDRA-14867 ? > Histogram overflow causes client timeouts and message drops > ----------------------------------------------------------- > > Key: CASSANDRA-19082 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19082 > Project: Cassandra > Issue Type: Bug > Reporter: Zbyszek Z > Priority: Normal > > Hi, > We have recently noticed that sometimes this exception happens on our > Cassandra cluster: > {code:java} > ERROR [ScheduledTasks:1] 2023-11-24 06:24:12,680 CassandraDaemon.java:244 - > Exception in thread Thread[ScheduledTasks:1,5,main] > java.lang.IllegalStateException: Unable to compute when histogram overflowed > at > org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoir$EstimatedHistogramReservoirSnapshot.getMean(DecayingEstimatedHistogramReservoir.java:472) > at > org.apache.cassandra.net.MessagingService.getDroppedMessagesLogs(MessagingService.java:1272) > at > org.apache.cassandra.net.MessagingService.logDroppedMessages(MessagingService.java:1244) > at > org.apache.cassandra.net.MessagingService.access$200(MessagingService.java:84) > at > org.apache.cassandra.net.MessagingService$4.run(MessagingService.java:512) > at > org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84) > at java.lang.Thread.run(Thread.java:750) > {code} > It happens on all 6 nodes at the same time. Also we see increased client > timeouts and dropped READ and READ_RESPONSE messages. Our Cassandra is > 3.11.16, 2 DC setup, 6 node in each DC. RF is 3. I have searched issues but > could not find exactly same issue causing messages to be dropped. Any > suggestion would be appreciated. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org