[ https://issues.apache.org/jira/browse/HBASE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798608#comment-16798608 ]
Sergey Shelukhin commented on HBASE-22079: ------------------------------------------ Replication logcleaner chore is leaking it... however what is not clear is why it doesn't get shut down by chore scheduler {noformat} 2019-03-15 14:51:50,804 INFO [master/:17000:becomeActiveMaster] zookeeper.RecoverableZooKeeper: Process identifier=replicationLogCleaner connecting to ZooKeeper ensemble= 2019-03-15 14:51:51,554 INFO [master/:17000:becomeActiveMaster] cleaner.LogCleaner: Creating OldWALs cleaners with size=2 {noformat} The ScheduledChore "was stopped" line for this chore is missing... and in fact there isn't any line for it even though it's scheduled as a chore. > master leaks ZK on shutdown and gets stuck because of netty threads if netty > socket is used > ------------------------------------------------------------------------------------------- > > Key: HBASE-22079 > URL: https://issues.apache.org/jira/browse/HBASE-22079 > Project: HBase > Issue Type: Bug > Reporter: Sergey Shelukhin > Assignee: Sergey Shelukhin > Priority: Major > > {noformat} > "master/...:17000:becomeActiveMaster-SendThread(...1)" #311 daemon prio=5 > os_prio=0 tid=0x0000000058c61800 nid=0x2dd0 waiting on condition > [0x0000000c477fe000] > java.lang.Thread.State: TIMED_WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000000c4a5b3c0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > at > java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522) > at > java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684) > at > org.apache.zookeeper.ClientCnxnSocketNetty.doTransport(ClientCnxnSocketNetty.java:232) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1146) > {noformat} > This causes a bunch of netty threads to also leak it looks like, and these > are not daemon (by design, apparently) -- This message was sent by Atlassian JIRA (v7.6.3#76005)