Yes, it is reproducible See client thread dump attached
Server node is just logging metrics, nothing more: [13:03:10,830][INFO][grid-timeout-worker-#23][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=d0ae4736, uptime=00:15:00.098] ^-- H/N/C [hosts=1, nodes=2, CPUs=2] ^-- CPU [cur=0.5%, avg=1.02%, GC=0%] ^-- PageMemory [pages=0] ^-- Heap [used=67MB, free=72.64%, comm=245MB] ^-- Non heap [used=43MB, free=97.12%, comm=44MB] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=6, qSize=0] ^-- Outbound messages queue [size=0] On Tue, Mar 6, 2018 at 12:29 PM, Stanislav Lukyanov <stanlukya...@gmail.com> wrote: > Hi Olexandr, > > > > Is it reproducible? Can you share a full thread dump? > > Also, are there any related messages in the logs (make sure you’re running > with `java -DIGNITE_QUEIT=false` or with `ignite.sh -v`)? > > > > Thanks, > > Stan > > > > *From: *Olexandr K <olexandr.kundire...@gmail.com> > *Sent: *6 марта 2018 г. 12:48 > *To: *user@ignite.apache.org > *Subject: *How to reconnect Ignite client after server node bounce? > > > > Hi Team, > > I tried the following scenario: > > 1) start local ignite server node > > 2) start client node and put some key/value pairs > > 3) bounce server node > > 4) put some more key/value pairs from client node > > > > at step (4) ignite put(..) operation just hanged > > I expected ignite client to take care of auto-reconnection or at least > fail-fast... > > How should I handle this? > > > > here is hanged thread's stack: > > Daemon Thread [async-worker-2] (Suspended) > Unsafe.park(boolean, long) line: not available [native method] > LockSupport.park() line: 304 > lock -> > GridFutureAdapter$ChainFuture<R,T>(GridFutureAdapter<R>).get0(boolean) > line: 177 > GridFutureAdapter$ChainFuture<R,T>(GridFutureAdapter<R>).get() line: > 140 > GridCacheAdapter$22.op(GridNearTxLocal) line: 2342 > GridCacheAdapter$22.op(GridNearTxLocal) line: 2340 > > GridDhtColocatedCache<K,V>(GridCacheAdapter<K,V>).syncOp(GridCacheAdapter<K,SyncOp<>>) > line: 4040 > GridDhtColocatedCache<K,V>(GridCacheAdapter<K,V>).put0(K, V, > CacheEntryPredicate) line: 2340 > GridDhtColocatedCache<K,V>(GridCacheAdapter<K,V>).put(K, V, > CacheEntryPredicate) line: 2321 > GridDhtColocatedCache<K,V>(GridCacheAdapter<K,V>).put(K, V) line: > 2298 > IgniteCacheProxyImpl<K,V>.put(K, V) line: 1005 > GatewayProtectedCacheProxy<K,V>.put(K, V) line: 872 > put -> PersonRegistrationController.lambda$2(String, String, String, > String) line: 106 > ... > > (I'm using ignite 2.3.0) > > > > BR, Oleksandr > > >
2018-03-06 13:00:12 Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.161-b12 mixed mode): "async-worker-6" #123 daemon prio=5 os_prio=0 tid=0x00000000011fd000 nid=0x5166 waiting on condition [0x00007f06d75f4000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2342) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2340) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4040) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2340) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2321) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2298) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1005) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:872) at com.company.lk.backend.ctl.person.PersonRegistrationController.lambda$2(PersonRegistrationController.java:107) at com.company.lk.backend.ctl.person.PersonRegistrationController$$Lambda$332/1484018980.get(Unknown Source) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - <0x00000000fb6c8948> (a java.util.concurrent.ThreadPoolExecutor$Worker) "async-worker-5" #122 daemon prio=5 os_prio=0 tid=0x00007f06f000a000 nid=0x5165 waiting on condition [0x00007f06d4caa000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2342) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2340) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4040) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2340) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2321) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2298) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1005) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:872) at com.company.lk.backend.ctl.person.PersonRegistrationController.lambda$2(PersonRegistrationController.java:107) at com.company.lk.backend.ctl.person.PersonRegistrationController$$Lambda$332/1484018980.get(Unknown Source) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - <0x00000000fb432380> (a java.util.concurrent.ThreadPoolExecutor$Worker) "async-worker-4" #120 daemon prio=5 os_prio=0 tid=0x00007f06e000d800 nid=0x5162 waiting on condition [0x00007f06d73f2000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f0f5bef8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "RMI Scheduler(0)" #111 daemon prio=9 os_prio=0 tid=0x00007f06fc096800 nid=0x5026 waiting on condition [0x00007f06bbc8a000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000feabc750> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "RMI TCP Accept-0" #109 daemon prio=9 os_prio=0 tid=0x00007f06f4020000 nid=0x5022 runnable [0x00007f06d48a7000] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:52) at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400) at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "Attach Listener" #108 daemon prio=9 os_prio=0 tid=0x00007f06f4008000 nid=0x5021 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - None "async-worker-3" #104 daemon prio=5 os_prio=0 tid=0x00007f07245bc800 nid=0x4fc3 waiting on condition [0x00007f06d49a7000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2342) at org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2340) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4040) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2340) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2321) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2298) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1005) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:872) at com.company.lk.backend.ctl.person.PersonRegistrationController.lambda$2(PersonRegistrationController.java:107) at com.company.lk.backend.ctl.person.PersonRegistrationController$$Lambda$332/1484018980.get(Unknown Source) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - <0x00000000fe939f38> (a java.util.concurrent.ThreadPoolExecutor$Worker) "async-worker-2" #98 daemon prio=5 os_prio=0 tid=0x00007f06dc0af000 nid=0x4f0d waiting on condition [0x00007f06bbd8b000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f0f5bef8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "async-worker-1" #96 daemon prio=5 os_prio=0 tid=0x00000000011ff800 nid=0x4f0a waiting on condition [0x00007f07099c8000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f0f5bef8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "DestroyJavaVM" #93 prio=5 os_prio=0 tid=0x00007f072400f000 nid=0x4e8e waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - None "https-jsse-nio-8080-AsyncTimeout" #91 daemon prio=5 os_prio=0 tid=0x00007f06d9030800 nid=0x4f01 waiting on condition [0x00007f06bc28c000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.coyote.AbstractProtocol$AsyncTimeout.run(AbstractProtocol.java:1133) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-Acceptor-0" #90 daemon prio=5 os_prio=0 tid=0x00007f06d902f000 nid=0x4f00 runnable [0x00007f06bc38d000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) - locked <0x00000000fe8a2cd8> (a java.lang.Object) at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:455) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-ClientPoller-1" #89 daemon prio=5 os_prio=0 tid=0x00007f06d902c800 nid=0x4eff runnable [0x00007f06bc48e000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000fe8e0c18> (a sun.nio.ch.Util$3) - locked <0x00000000fe8e0c08> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fe8a3350> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:793) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-ClientPoller-0" #88 daemon prio=5 os_prio=0 tid=0x00007f06d9012000 nid=0x4efe runnable [0x00007f06bc58f000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000fe8e13b0> (a sun.nio.ch.Util$3) - locked <0x00000000fe8e13a0> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fe8e1158> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:793) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-10" #87 daemon prio=5 os_prio=0 tid=0x00007f06d8ff6000 nid=0x4efd waiting on condition [0x00007f06bc690000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-9" #86 daemon prio=5 os_prio=0 tid=0x00007f06d8ff4000 nid=0x4efc waiting on condition [0x00007f06bc791000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-8" #85 daemon prio=5 os_prio=0 tid=0x00007f06d8ff2000 nid=0x4efb waiting on condition [0x00007f06bc892000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-7" #84 daemon prio=5 os_prio=0 tid=0x00007f06d8ff0000 nid=0x4efa waiting on condition [0x00007f06bc993000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-6" #83 daemon prio=5 os_prio=0 tid=0x00007f06d8fee000 nid=0x4ef9 waiting on condition [0x00007f06bca94000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-5" #82 daemon prio=5 os_prio=0 tid=0x00007f06d8fec000 nid=0x4ef8 waiting on condition [0x00007f06bcb95000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-4" #81 daemon prio=5 os_prio=0 tid=0x00007f06d8fea000 nid=0x4ef7 waiting on condition [0x00007f06bcc96000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-3" #80 daemon prio=5 os_prio=0 tid=0x00007f06d8fe8000 nid=0x4ef6 waiting on condition [0x00007f06bcd97000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-2" #79 daemon prio=5 os_prio=0 tid=0x00007f06d8fe6800 nid=0x4ef5 waiting on condition [0x00007f06bce98000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "https-jsse-nio-8080-exec-1" #78 daemon prio=5 os_prio=0 tid=0x00007f06d8fe5800 nid=0x4ef4 waiting on condition [0x00007f06bcf99000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fe88d138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "NioBlockingSelector.BlockPoller-1" #77 daemon prio=5 os_prio=0 tid=0x00007f06d8fbd800 nid=0x4ef3 runnable [0x00007f07096c7000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000fe8e1e00> (a sun.nio.ch.Util$3) - locked <0x00000000fe8e1df0> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fe8e1cc8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:298) Locked ownable synchronizers: - None "Live Reload Server" #76 daemon prio=5 os_prio=0 tid=0x00007f06d8f8f800 nid=0x4eed runnable [0x00007f06d41a4000] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at org.springframework.boot.devtools.livereload.LiveReloadServer.acceptConnections(LiveReloadServer.java:146) at org.springframework.boot.devtools.livereload.LiveReloadServer$$Lambda$310/384817350.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "File Watcher" #74 daemon prio=5 os_prio=0 tid=0x00007f06d8f8d800 nid=0x4eeb waiting on condition [0x00007f0708bc6000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.scan(FileSystemWatcher.java:253) at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.run(FileSystemWatcher.java:243) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "kafka-producer-network-thread | producer-1" #73 daemon prio=5 os_prio=0 tid=0x00007f06d8cec000 nid=0x4ee4 runnable [0x00007f06d44a5000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f1c8e980> (a sun.nio.ch.Util$3) - locked <0x00000000f1c8e970> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f1c8e848> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.kafka.common.network.Selector.select(Selector.java:657) at org.apache.kafka.common.network.Selector.poll(Selector.java:383) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:460) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "KafkaReceiver-1679527137" #72 prio=5 os_prio=0 tid=0x00007f06d8413800 nid=0x4ee3 runnable [0x00007f06d47a6000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f1c40dc0> (a sun.nio.ch.Util$3) - locked <0x00000000f1c40db0> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f1c40138> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.kafka.common.network.Selector.select(Selector.java:657) at org.apache.kafka.common.network.Selector.poll(Selector.java:383) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:460) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:238) - locked <0x00000000f1c63488> (a org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:214) at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1156) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1103) at com.company.lk.backend.core.KafkaReceiver.start(KafkaReceiver.java:59) at com.company.lk.backend.core.KafkaReceiver.lambda$0(KafkaReceiver.java:46) at com.company.lk.backend.core.KafkaReceiver$$Lambda$264/1817594127.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "session-timeout-worker-#32" #56 daemon prio=5 os_prio=0 tid=0x00007f06d83df000 nid=0x4ede waiting on condition [0x00007f06d4aa9000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.ignite.internal.processors.rest.GridRestProcessor$4.body(GridRestProcessor.java:437) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "exchange-worker-#35" #66 prio=5 os_prio=0 tid=0x00007f06d83da000 nid=0x4eda waiting on condition [0x00007f06d51ac000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:217) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:159) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2289) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "ignite-update-notifier-timer" #65 daemon prio=5 os_prio=0 tid=0x00007f06d82dc800 nid=0x4ed9 in Object.wait() [0x00007f06d52ad000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000f1b8c168> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:552) - locked <0x00000000f1b8c168> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505) Locked ownable synchronizers: - None "upd-ver-checker" #64 daemon prio=5 os_prio=0 tid=0x00007f06d82d9800 nid=0x4ed8 waiting on condition [0x00007f06d54af000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.ignite.internal.processors.cluster.GridUpdateNotifier$1.run(GridUpdateNotifier.java:115) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "disco-event-worker-#34" #63 prio=5 os_prio=0 tid=0x00007f06d82d6800 nid=0x4ed7 waiting on condition [0x00007f06d53ae000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f151bb10> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2552) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2534) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "tcp-client-disco-msg-worker-#4" #60 prio=5 os_prio=0 tid=0x00007f06d8ca8000 nid=0x4ed2 waiting on condition [0x00007f06d55b0000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f17fe380> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492) at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680) at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1533) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Locked ownable synchronizers: - None "tcp-client-disco-sock-reader-#3" #59 prio=5 os_prio=0 tid=0x00007f06d8ca5000 nid=0x4ed1 runnable [0x00007f06d56b1000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) - locked <0x00000000f17fd8f8> (a java.io.BufferedInputStream) at org.apache.ignite.marshaller.jdk.JdkMarshallerInputStreamWrapper.read(JdkMarshallerInputStreamWrapper.java:53) at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2653) at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2669) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3146) at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:858) at java.io.ObjectInputStream.<init>(ObjectInputStream.java:354) at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.<init>(JdkMarshallerObjectInputStream.java:39) at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:119) at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94) at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9740) at org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketReader.body(ClientImpl.java:1001) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Locked ownable synchronizers: - None "tcp-client-disco-sock-writer-#2" #58 prio=5 os_prio=0 tid=0x00007f06d8ca3800 nid=0x4ed0 in Object.wait() [0x00007f06d57b2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketWriter.body(ClientImpl.java:1219) - locked <0x00000000f17fd058> (a java.lang.Object) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Locked ownable synchronizers: - None "grid-data-loader-flusher-#33" #57 prio=5 os_prio=0 tid=0x00007f06d8c85800 nid=0x4ecf waiting on condition [0x00007f06d5ab3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f180c830> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.DelayQueue.take(DelayQueue.java:211) at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:108) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "grid-nio-worker-client-listener-1-#31" #55 prio=5 os_prio=0 tid=0x00007f06d8c78800 nid=0x4ece runnable [0x00007f06d5bb4000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f180db70> (a org.apache.ignite.internal.util.nio.SelectedSelectionKeySet) - locked <0x00000000f180dbd0> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f180dac8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2043) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1717) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "grid-nio-worker-client-listener-0-#30" #54 prio=5 os_prio=0 tid=0x00007f06d8c77000 nid=0x4ecd runnable [0x00007f06d5cb5000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f194aca0> (a org.apache.ignite.internal.util.nio.SelectedSelectionKeySet) - locked <0x00000000f194bd10> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f194abf8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2043) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1717) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "nio-acceptor-#29" #53 prio=5 os_prio=0 tid=0x00007f06d8c76000 nid=0x4ecc runnable [0x00007f06d5db6000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f180e518> (a sun.nio.ch.Util$3) - locked <0x00000000f180e508> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f180e0f0> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2772) at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2744) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "TcpDiscoverySpi.timer" #52 prio=5 os_prio=0 tid=0x00007f06d8ae0800 nid=0x4ecb in Object.wait() [0x00007f06d60b7000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.util.TimerThread.mainLoop(Timer.java:552) - locked <0x00000000f17fda20> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505) Locked ownable synchronizers: - None "tcp-comm-worker-#1" #51 prio=5 os_prio=0 tid=0x00007f06d8ab5800 nid=0x4eca waiting on condition [0x00007f06d63b8000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f129bad8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4037) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) Locked ownable synchronizers: - None "grid-nio-worker-tcp-comm-3-#28" #46 prio=5 os_prio=0 tid=0x00007f06d8ab4000 nid=0x4ec9 runnable [0x00007f06d64b9000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f1226cd0> (a org.apache.ignite.internal.util.nio.SelectedSelectionKeySet) - locked <0x00000000f1226d30> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f1226c28> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2043) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1717) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "grid-nio-worker-tcp-comm-2-#27" #45 prio=5 os_prio=0 tid=0x00007f06d8ab1000 nid=0x4ec8 runnable [0x00007f06d6dee000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f1227430> (a org.apache.ignite.internal.util.nio.SelectedSelectionKeySet) - locked <0x00000000f1227490> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f1227388> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2043) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1717) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "grid-nio-worker-tcp-comm-1-#26" #44 prio=5 os_prio=0 tid=0x00007f06d8aae000 nid=0x4ec7 runnable [0x00007f06d6ced000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f1265b60> (a org.apache.ignite.internal.util.nio.SelectedSelectionKeySet) - locked <0x00000000f1266bd0> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f1265ab8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2043) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1717) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "grid-nio-worker-tcp-comm-0-#25" #43 prio=5 os_prio=0 tid=0x00007f06d8aac800 nid=0x4ec6 runnable [0x00007f06d6eef000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f12672b8> (a org.apache.ignite.internal.util.nio.SelectedSelectionKeySet) - locked <0x00000000f1267318> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f1267210> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2043) at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1717) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "nio-acceptor-#24" #42 prio=5 os_prio=0 tid=0x00007f06d8aa9000 nid=0x4ec5 runnable [0x00007f06d6bec000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) - locked <0x00000000f1226530> (a sun.nio.ch.Util$3) - locked <0x00000000f1226520> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000f12260f8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2772) at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2744) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "grid-timeout-worker-#23" #41 prio=5 os_prio=0 tid=0x00007f06d8a07000 nid=0x4ec0 in Object.wait() [0x00007f06d6ff0000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:195) - locked <0x00000000f128f6f8> (a java.lang.Object) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "ignite-clock" #40 daemon prio=10 os_prio=0 tid=0x00007f06d89fd000 nid=0x4ebf sleeping[0x00007f06d70f1000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.ignite.internal.util.IgniteUtils$22.run(IgniteUtils.java:3263) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-7-#16" #33 prio=5 os_prio=0 tid=0x00007f06d891d800 nid=0x4eb8 waiting on condition [0x00007f06d79f8000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-6-#15" #32 prio=5 os_prio=0 tid=0x00007f06d891c000 nid=0x4eb7 waiting on condition [0x00007f06d7af9000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-5-#14" #31 prio=5 os_prio=0 tid=0x00007f06d891a000 nid=0x4eb6 waiting on condition [0x00007f06d7bfa000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-4-#13" #30 prio=5 os_prio=0 tid=0x00007f06d8918800 nid=0x4eb5 waiting on condition [0x00007f06d7cfb000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-3-#12" #29 prio=5 os_prio=0 tid=0x00007f06d8917000 nid=0x4eb4 waiting on condition [0x00007f06d7dfc000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-2-#11" #28 prio=5 os_prio=0 tid=0x00007f06d8915000 nid=0x4eb3 waiting on condition [0x00007f06d7efd000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-1-#10" #27 prio=5 os_prio=0 tid=0x00007f06d8913800 nid=0x4eb2 waiting on condition [0x00007f06d7ffe000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "data-streamer-stripe-0-#9" #26 prio=5 os_prio=0 tid=0x00007f06d8911800 nid=0x4eb1 waiting on condition [0x00007f06e81f3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-7-#8" #25 prio=5 os_prio=0 tid=0x00007f06d8910000 nid=0x4eb0 waiting on condition [0x00007f06e82f4000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-6-#7" #24 prio=5 os_prio=0 tid=0x00007f06d890e000 nid=0x4eaf waiting on condition [0x00007f06e83f5000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-5-#6" #23 prio=5 os_prio=0 tid=0x00007f06d890c000 nid=0x4eae waiting on condition [0x00007f06e84f6000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-4-#5" #22 prio=5 os_prio=0 tid=0x00007f06d890a000 nid=0x4ead waiting on condition [0x00007f06e85f7000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-3-#4" #21 prio=5 os_prio=0 tid=0x00007f06d8908000 nid=0x4eac waiting on condition [0x00007f06e86f8000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-2-#3" #20 prio=5 os_prio=0 tid=0x00007f06d8906000 nid=0x4eab waiting on condition [0x00007f06e87f9000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-1-#2" #19 prio=5 os_prio=0 tid=0x00007f06d8904800 nid=0x4eaa waiting on condition [0x00007f06e88fa000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "sys-stripe-0-#1" #18 prio=5 os_prio=0 tid=0x00007f06d8902000 nid=0x4ea9 waiting on condition [0x00007f06e89fb000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "container-0" #17 prio=5 os_prio=0 tid=0x00007f06d82df800 nid=0x4ea6 waiting on condition [0x00007f06e8efc000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.catalina.core.StandardServer.await(StandardServer.java:427) at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run(TomcatWebServer.java:181) Locked ownable synchronizers: - None "ContainerBackgroundProcessor[StandardEngine[Tomcat]]" #16 daemon prio=5 os_prio=0 tid=0x00007f06d82de800 nid=0x4ea5 waiting on condition [0x00007f06e8ffd000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1355) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None "Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007f07240c0800 nid=0x4e98 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - None "C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f07240bd800 nid=0x4e97 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - None "C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f07240bb000 nid=0x4e96 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - None "Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f07240b9800 nid=0x4e95 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - None "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f0724085000 nid=0x4e94 in Object.wait() [0x00007f070b4d2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) - locked <0x00000000f00414e0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) Locked ownable synchronizers: - None "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f0724080800 nid=0x4e93 in Object.wait() [0x00007f070b5d3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) - locked <0x00000000f00edda0> (a java.lang.ref.Reference$Lock) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) Locked ownable synchronizers: - None "VM Thread" os_prio=0 tid=0x00007f0724079000 nid=0x4e92 runnable "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007f0724024800 nid=0x4e90 runnable "GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007f0724026000 nid=0x4e91 runnable "VM Periodic Task Thread" os_prio=0 tid=0x00007f07240c6000 nid=0x4e99 waiting on condition JNI global references: 991