rdhabalia opened a new pull request #3633: [pulsar-broker] Fix deadlock: add zk-operation timeout for blocking call on zk-cache URL: https://github.com/apache/pulsar/pull/3633 ### Motivation have seen this deadlock issue multiple times in the brokers while creating topics and adding consumers. ``` "pulsar-io-21-15" #340 prio=5 os_prio=0 tid=0x00002af27c020800 nid=0x263d waiting on condition [0x00002aefdfb09000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000006960b8018> (a java.util.concurrent.CompletableFuture$Signaller) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693) at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323) at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) at org.apache.pulsar.zookeeper.ZooKeeperDataCache.get(ZooKeeperDataCache.java:94) at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.isConsumersExceededOnTopic(PersistentDispatcherMultipleConsumers.java:148) at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.addConsumer(PersistentDispatcherMultipleConsumers.java:129) - locked <0x000000058afd2dd0> (a org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers) at org.apache.pulsar.broker.service.persistent.PersistentSubscription.addConsumer(PersistentSubscription.java:139) - locked <0x000000058afd2ed8> (a org.apache.pulsar.broker.service.persistent.PersistentSubscription) at org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$9(PersistentTopic.java:523) at org.apache.pulsar.broker.service.persistent.PersistentTopic$$Lambda$309/503840967.accept(Unknown Source) at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656) at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:669) at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:1997) at org.apache.pulsar.broker.service.persistent.PersistentTopic.subscribe(PersistentTopic.java:519) at org.apache.pulsar.broker.service.ServerCnx.lambda$22(ServerCnx.java:609) at org.apache.pulsar.broker.service.ServerCnx$$Lambda$299/1090387422.apply(Unknown Source) at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981) at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124) at org.apache.pulsar.broker.service.ServerCnx.lambda$18(ServerCnx.java:592) at org.apache.pulsar.broker.service.ServerCnx$$Lambda$298/1756865666.apply(Unknown Source) at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602) at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:614) at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1983) at org.apache.pulsar.broker.service.ServerCnx.lambda$6(ServerCnx.java:550) at org.apache.pulsar.broker.service.ServerCnx$$Lambda$294/380312495.apply(Unknown Source) at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602) at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:614) at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1983) at org.apache.pulsar.broker.service.ServerCnx.handleSubscribe(ServerCnx.java:539) ``` ### Modification To avoid deadlock on zk-cache's blocking call, add zk-operation timeout.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
