[
https://issues.apache.org/jira/browse/HBASE-13093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14335771#comment-14335771
]
Andrey Stepachev commented on HBASE-13093:
------------------------------------------
Looks like HashedWheelTimer in AsyncRpcClient uses
Executors#defaultThreadFactory which in turn non-daemon. That can happen if
some timeout was not canceled. [~jurmous], can you take a look, can that be
related to some not canceled timeout? thanks.
> Local mode HBase instance doesn't shut down.
> --------------------------------------------
>
> Key: HBASE-13093
> URL: https://issues.apache.org/jira/browse/HBASE-13093
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Elliott Clark
>
> {code}bin/start-hbase.sh{code}
> {code}bin/stop-hbase.sh{code}
> That hangs forever. Here's the jstacks:
> {code}2015-02-24 16:37:55
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.60-b09 mixed mode):
> "Attach Listener" daemon prio=5 tid=0x00007fb130813800 nid=0xfd07 waiting on
> condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "DestroyJavaVM" prio=5 tid=0x00007fb12ba7c800 nid=0x1303 waiting on condition
> [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "pool-5-thread-1" prio=5 tid=0x00007fb12bb88800 nid=0x19903 waiting on
> condition [0x0000000121a1b000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:461)
> at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:360)
> at java.lang.Thread.run(Thread.java:745)
> "HBase-Metrics2-1" daemon prio=5 tid=0x00007fb12c040000 nid=0x19703 waiting
> on condition [0x0000000121918000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000724cc9780> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "snapshot-hfile-cleaner-cache-refresher" daemon prio=5 tid=0x00007fb12bc91000
> nid=0x18703 in Object.wait() [0x000000012160f000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000724caa588> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:552)
> - locked <0x0000000724caa588> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
> "snapshot-log-cleaner-cache-refresher" daemon prio=5 tid=0x00007fb12bbc8000
> nid=0x18503 in Object.wait() [0x000000012150c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000724deb178> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:552)
> - locked <0x0000000724deb178> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
> "localhost:57343.activeMasterManager-EventThread" daemon prio=5
> tid=0x00007fb12c072000 nid=0x18303 waiting on condition [0x0000000121409000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000724f10150> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
> "localhost:57343.activeMasterManager-SendThread(fe80:0:0:0:0:0:0:1%1:2181)"
> daemon prio=5 tid=0x00007fb12c053000 nid=0x18103 waiting on condition
> [0x0000000121306000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
> "Timer for 'HBase' metrics system" daemon prio=5 tid=0x00007fb12de54800
> nid=0x7903 in Object.wait() [0x00000001187df000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000724b4f150> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:552)
> - locked <0x0000000724b4f150> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
> "Service Thread" daemon prio=5 tid=0x00007fb12e002000 nid=0x5503 runnable
> [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=5 tid=0x00007fb12d92e000 nid=0x5303 waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=5 tid=0x00007fb12b832800 nid=0x5103 waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=5 tid=0x00007fb12b80f800 nid=0x4f03 runnable
> [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Surrogate Locker Thread (Concurrent GC)" daemon prio=5
> tid=0x00007fb12d90c000 nid=0x400b waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=5 tid=0x00007fb12b824800 nid=0x3b03 in Object.wait()
> [0x00000001145b9000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000007247cf250> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x00000007247cf250> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
> "Reference Handler" daemon prio=5 tid=0x00007fb12d902800 nid=0x3903 in
> Object.wait() [0x00000001144b6000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000007247c43a0> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x00000007247c43a0> (a java.lang.ref.Reference$Lock)
> "VM Thread" prio=5 tid=0x00007fb12b824000 nid=0x3703 runnable
> "Gang worker#0 (Parallel GC Threads)" prio=5 tid=0x00007fb12d801000
> nid=0x2103 runnable
> "Gang worker#1 (Parallel GC Threads)" prio=5 tid=0x00007fb12d80d800
> nid=0x2303 runnable
> "Gang worker#2 (Parallel GC Threads)" prio=5 tid=0x00007fb12d80e000
> nid=0x2503 runnable
> "Gang worker#3 (Parallel GC Threads)" prio=5 tid=0x00007fb12d80e800
> nid=0x2703 runnable
> "Gang worker#4 (Parallel GC Threads)" prio=5 tid=0x00007fb12d80f800
> nid=0x2903 runnable
> "Gang worker#5 (Parallel GC Threads)" prio=5 tid=0x00007fb12d810000
> nid=0x2b03 runnable
> "Gang worker#6 (Parallel GC Threads)" prio=5 tid=0x00007fb12d810800
> nid=0x2d03 runnable
> "Gang worker#7 (Parallel GC Threads)" prio=5 tid=0x00007fb12d811000
> nid=0x2f03 runnable
> "Concurrent Mark-Sweep GC Thread" prio=5 tid=0x00007fb12d8d3800 nid=0x3503
> runnable
> "Gang worker#0 (Parallel CMS Threads)" prio=5 tid=0x00007fb12d8d2800
> nid=0x3103 runnable
> "Gang worker#1 (Parallel CMS Threads)" prio=5 tid=0x00007fb12d8d3000
> nid=0x3303 runnable
> "VM Periodic Task Thread" prio=5 tid=0x00007fb12d92d800 nid=0x5703 waiting on
> condition
> JNI global references: 177
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
