Shashikant Banerjee created RATIS-790: -----------------------------------------
Summary: GrpcLogAppenders on Ratis leader block on each other Key: RATIS-790 URL: https://issues.apache.org/jira/browse/RATIS-790 Project: Ratis Issue Type: Bug Components: server Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee Fix For: 0.5.0 {code:java} "org.apache.ratis.server.impl.LogAppender$AppenderDaemon$$Lambda$369/802019944@2f87467e" #152 daemon prio=5 os_prio=0 tid=0x00007f6648271000 nid=0x7aeb waiting on condition [0x00007f65d7803000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000003d82f4208> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.offer(ScheduledThreadPoolExecutor.java:1010) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.add(ScheduledThreadPoolExecutor.java:1037) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.add(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:328) at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533) at org.apache.ratis.util.TimeoutScheduler$Scheduler.schedule(TimeoutScheduler.java:83) at org.apache.ratis.util.TimeoutScheduler.onTimeout(TimeoutScheduler.java:155) - locked <0x00000003d7ecd4e8> (a org.apache.ratis.util.TimeoutScheduler) at org.apache.ratis.util.TimeoutScheduler.onTimeout(TimeoutScheduler.java:138) at org.apache.ratis.util.TimeoutScheduler.onTimeout(TimeoutScheduler.java:191) at org.apache.ratis.grpc.server.GrpcLogAppender.sendRequest(GrpcLogAppender.java:203) at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:194) at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:121) at org.apache.ratis.server.impl.LogAppender$AppenderDaemon.run(LogAppender.java:77) at org.apache.ratis.server.impl.LogAppender$AppenderDaemon$$Lambda$369/802019944.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) "org.apache.ratis.server.impl.LogAppender$AppenderDaemon$$Lambda$369/802019944@19778529" #150 daemon prio=5 os_prio=0 tid=0x00007f664826f800 nid=0x7aea waiting for monitor entry [0x00007f65d7904000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.ratis.util.TimeoutScheduler.onTimeout(TimeoutScheduler.java:151) - waiting to lock <0x00000003d7ecd4e8> (a org.apache.ratis.util.TimeoutScheduler) at org.apache.ratis.util.TimeoutScheduler.onTimeout(TimeoutScheduler.java:138) at org.apache.ratis.util.TimeoutScheduler.onTimeout(TimeoutScheduler.java:191) at org.apache.ratis.grpc.server.GrpcLogAppender.sendRequest(GrpcLogAppender.java:203) at org.apache.ratis.grpc.server.GrpcLogAppender.appendLog(GrpcLogAppender.java:194) at org.apache.ratis.grpc.server.GrpcLogAppender.runAppenderImpl(GrpcLogAppender.java:121) at org.apache.ratis.server.impl.LogAppender$AppenderDaemon.run(LogAppender.java:77) at org.apache.ratis.server.impl.LogAppender$AppenderDaemon$$Lambda$369/802019944.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) {code} The GrpcLogAppenders, one for each follower on a ratis leader seem to block to each other because both of them share same instance of TimeoutScheduler . -- This message was sent by Atlassian Jira (v8.3.4#803005)