[ https://issues.apache.org/jira/browse/IGNITE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746722#comment-16746722 ]
Dmitriy Govorukhin commented on IGNITE-10974: --------------------------------------------- I guess that problem in a test because they were run on windows agent. I started new re-run. > Node may hangs if an exception is throw from > PageMemoryImpl.beforeReleaseWrite() > -------------------------------------------------------------------------------- > > Key: IGNITE-10974 > URL: https://issues.apache.org/jira/browse/IGNITE-10974 > Project: Ignite > Issue Type: Bug > Reporter: Dmitriy Govorukhin > Assignee: Dmitriy Govorukhin > Priority: Critical > Fix For: 2.8 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > There are situations when a node may hang during stopping. Bug in the > PageMemoryImpl.writeUnlockPage(), this method invokes > PageMemoryImpl.beforeReleaseWrite() which may throw a exception and > writeUnlockPage does not release page lock, after that checkpoint try to do > final checkpoint (if stop was gracefully) and next access for this page hangs > on obtain write lock. > {code:java} > [2019-01-17 14:35:15,953][WARN ][main][root] Thread dump at 2019/01/17 > 14:35:15 UTC > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="sys-#857%failure.IoomFailureHandlerTest0%", id=931, > state=TIMED_WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4339baec, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="sys-#856%failure.IoomFailureHandlerTest0%", id=930, > state=TIMED_WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4339baec, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="sys-#855%failure.IoomFailureHandlerTest0%", id=929, > state=TIMED_WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4339baec, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="sys-#854%failure.IoomFailureHandlerTest0%", id=928, > state=TIMED_WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4339baec, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="sys-#853%failure.IoomFailureHandlerTest0%", id=927, > state=TIMED_WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4339baec, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="sys-#852%failure.IoomFailureHandlerTest0%", id=926, > state=TIMED_WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4339baec, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#834%failure.IoomFailureHandlerTest1%", id=908, > state=WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@62db3891, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.waitAcquireWriteLock(OffheapReadWriteLock.java:488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.writeLock(OffheapReadWriteLock.java:229) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLockPage(PageMemoryImpl.java:1488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:456) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:451) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.tree.util.PageHandler.writeLock(PageHandler.java:415) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.DataStructure.writeLock(DataStructure.java:173) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.freelist.PagesList.saveMetadata(PagesList.java:309) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.saveStoreMetadata(GridCacheOffheapManager.java:258) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.lambda$syncMetadata$2(GridCacheOffheapManager.java:227) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager$$Lambda$205/2012477277.run(Unknown > Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.lambda$wrapIgniteFuture$3(IgniteUtils.java:10937) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils$$Lambda$203/1785994943.run(Unknown Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Locked synchronizers: > [17:35:15]W: [org.apache.ignite:ignite-core] > java.util.concurrent.ThreadPoolExecutor$Worker@65e22807 > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#833%failure.IoomFailureHandlerTest1%", id=907, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@61cd1c71, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.waitAcquireWriteLock(OffheapReadWriteLock.java:488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.writeLock(OffheapReadWriteLock.java:229) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLockPage(PageMemoryImpl.java:1488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:456) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:451) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.tree.util.PageHandler.writeLock(PageHandler.java:415) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.DataStructure.writeLock(DataStructure.java:173) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.freelist.PagesList.saveMetadata(PagesList.java:309) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.saveStoreMetadata(GridCacheOffheapManager.java:258) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.lambda$syncMetadata$2(GridCacheOffheapManager.java:227) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager$$Lambda$205/2012477277.run(Unknown > Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.lambda$wrapIgniteFuture$3(IgniteUtils.java:10937) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils$$Lambda$203/1785994943.run(Unknown Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Locked synchronizers: > [17:35:15]W: [org.apache.ignite:ignite-core] > java.util.concurrent.ThreadPoolExecutor$Worker@5406163f > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#832%failure.IoomFailureHandlerTest1%", id=906, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6724cdec, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.waitAcquireWriteLock(OffheapReadWriteLock.java:488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.writeLock(OffheapReadWriteLock.java:229) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLockPage(PageMemoryImpl.java:1488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:456) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:451) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.tree.util.PageHandler.writeLock(PageHandler.java:415) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.DataStructure.writeLock(DataStructure.java:173) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.freelist.PagesList.saveMetadata(PagesList.java:309) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.saveStoreMetadata(GridCacheOffheapManager.java:258) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.lambda$syncMetadata$2(GridCacheOffheapManager.java:227) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager$$Lambda$205/2012477277.run(Unknown > Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.lambda$wrapIgniteFuture$3(IgniteUtils.java:10937) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils$$Lambda$203/1785994943.run(Unknown Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Locked synchronizers: > [17:35:15]W: [org.apache.ignite:ignite-core] > java.util.concurrent.ThreadPoolExecutor$Worker@773048dc > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#831%failure.IoomFailureHandlerTest1%", id=905, > state=WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6944e53e, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.waitAcquireWriteLock(OffheapReadWriteLock.java:488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.OffheapReadWriteLock.writeLock(OffheapReadWriteLock.java:229) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLockPage(PageMemoryImpl.java:1488) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:456) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.pagemem.PageMemoryImpl.writeLock(PageMemoryImpl.java:451) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.tree.util.PageHandler.writeLock(PageHandler.java:415) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.DataStructure.writeLock(DataStructure.java:173) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.freelist.PagesList.saveMetadata(PagesList.java:309) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.saveStoreMetadata(GridCacheOffheapManager.java:258) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager.lambda$syncMetadata$2(GridCacheOffheapManager.java:227) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheOffheapManager$$Lambda$205/2012477277.run(Unknown > Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.lambda$wrapIgniteFuture$3(IgniteUtils.java:10937) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils$$Lambda$203/1785994943.run(Unknown Source) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Locked synchronizers: > [17:35:15]W: [org.apache.ignite:ignite-core] > java.util.concurrent.ThreadPoolExecutor$Worker@49173e64 > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#324%failure.IoomFailureHandlerTest0%", id=392, > state=WAITING, blockCnt=2, waitCnt=13] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52e04737, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#322%failure.IoomFailureHandlerTest0%", id=390, > state=WAITING, blockCnt=2, waitCnt=32] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52e04737, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#320%failure.IoomFailureHandlerTest0%", id=387, > state=WAITING, blockCnt=3, waitCnt=16] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52e04737, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="checkpoint-runner-#317%failure.IoomFailureHandlerTest0%", id=385, > state=WAITING, blockCnt=2, waitCnt=16] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52e04737, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="db-checkpoint-thread-#316%failure.IoomFailureHandlerTest1%", id=384, > state=WAITING, blockCnt=17, waitCnt=11] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.notifyCheckpointLisnteners(GridCacheDatabaseSharedManager.java:3959) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3786) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3342) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3223) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Locked synchronizers: > [17:35:15]W: [org.apache.ignite:ignite-core] > java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@5bc7e78e > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="db-checkpoint-thread-#315%failure.IoomFailureHandlerTest0%", id=383, > state=TIMED_WAITING, blockCnt=8, waitCnt=11] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer@1a4d1ab7, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.waitCheckpointEvent(GridCacheDatabaseSharedManager.java:3745) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3205) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="wal-segment-syncer-#298%failure.IoomFailureHandlerTest1%", id=366, > state=TIMED_WAITING, blockCnt=0, waitCnt=596] > [17:35:15]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7625) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.filehandle.FileHandleManagerImpl$WalSegmentSyncer.body(FileHandleManagerImpl.java:596) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="wal-file-archiver%failure.IoomFailureHandlerTest1-#297%failure.IoomFailureHandlerTest1%", > id=365, state=WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage@546e61d5, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Object.wait(Object.java:502) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage.awaitSegment(SegmentCurrentStateStorage.java:73) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage.waitNextSegmentForArchivation(SegmentCurrentStateStorage.java:90) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.aware.SegmentAware.waitNextSegmentForArchivation(SegmentAware.java:79) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.java:1664) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="wal-segment-syncer-#296%failure.IoomFailureHandlerTest0%", id=364, > state=TIMED_WAITING, blockCnt=0, waitCnt=596] > [17:35:15]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7625) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.filehandle.FileHandleManagerImpl$WalSegmentSyncer.body(FileHandleManagerImpl.java:596) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="wal-file-archiver%failure.IoomFailureHandlerTest0-#295%failure.IoomFailureHandlerTest0%", > id=363, state=WAITING, blockCnt=0, waitCnt=1] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage@4674d90, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Object.wait(Object.java:502) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage.awaitSegment(SegmentCurrentStateStorage.java:73) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.aware.SegmentCurrentStateStorage.waitNextSegmentForArchivation(SegmentCurrentStateStorage.java:90) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.aware.SegmentAware.waitNextSegmentForArchivation(SegmentAware.java:79) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.java:1664) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="disco-event-worker-#292%failure.IoomFailureHandlerTest1%", id=359, > state=WAITING, blockCnt=0, waitCnt=593] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@60783105, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2884) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2854) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-ip-finder-cleaner-#39%failure.IoomFailureHandlerTest1%", > id=358, state=TIMED_WAITING, blockCnt=0, waitCnt=5] > [17:35:15]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$IpFinderCleaner.body(ServerImpl.java:1962) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-sock-reader-#38%failure.IoomFailureHandlerTest0%", id=357, > state=RUNNABLE, blockCnt=2, waitCnt=0] > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.socketRead0(Native Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.socketRead(SocketInputStream.java:116) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.read(SocketInputStream.java:171) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.read(SocketInputStream.java:141) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.BufferedInputStream.fill(BufferedInputStream.java:246) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.BufferedInputStream.read1(BufferedInputStream.java:286) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.BufferedInputStream.read(BufferedInputStream.java:345) > [17:35:15]W: [org.apache.ignite:ignite-core] - locked > java.io.BufferedInputStream@1292071f > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.jdk.JdkMarshallerInputStreamWrapper.read(JdkMarshallerInputStreamWrapper.java:53) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2663) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2679) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream.<init>(ObjectInputStream.java:358) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.jdk.JdkMarshallerObjectInputStream.<init>(JdkMarshallerObjectInputStream.java:43) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:136) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:93) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.unmarshal(IgniteUtils.java:9928) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:6291) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-sock-reader-#37%failure.IoomFailureHandlerTest1%", id=356, > state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.socketRead0(Native Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.socketRead(SocketInputStream.java:116) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.read(SocketInputStream.java:171) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.SocketInputStream.read(SocketInputStream.java:141) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.BufferedInputStream.fill(BufferedInputStream.java:246) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.BufferedInputStream.read1(BufferedInputStream.java:286) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.BufferedInputStream.read(BufferedInputStream.java:345) > [17:35:15]W: [org.apache.ignite:ignite-core] - locked > java.io.BufferedInputStream@35d5ac51 > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.jdk.JdkMarshallerInputStreamWrapper.read(JdkMarshallerInputStreamWrapper.java:53) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2663) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2679) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.io.ObjectInputStream.<init>(ObjectInputStream.java:358) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.jdk.JdkMarshallerObjectInputStream.<init>(JdkMarshallerObjectInputStream.java:43) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:136) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:93) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.unmarshal(IgniteUtils.java:9928) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:6291) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-srvr-#35%failure.IoomFailureHandlerTest1%", id=354, > state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.PlainSocketImpl.socketAccept(Native Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.ServerSocket.implAccept(ServerSocket.java:545) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.net.ServerSocket.accept(ServerSocket.java:513) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5890) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5808) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-msg-worker-#34%failure.IoomFailureHandlerTest1%", id=353, > state=TIMED_WAITING, blockCnt=1, waitCnt=29927] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@b606cb6, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7228) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2715) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7164) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="disco-notifier-worker-#291%failure.IoomFailureHandlerTest1%", id=352, > state=WAITING, blockCnt=0, waitCnt=595] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2a3194c6, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2689) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2734) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="grid-sharedfs-timeout-worker-#33%failure.IoomFailureHandlerTest1%", > id=341, state=TIMED_WAITING, blockCnt=0, waitCnt=60] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.lang.Object@2819c460, ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.checkpoint.sharedfs.SharedFsTimeoutTask.body(SharedFsTimeoutTask.java:99) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-comm-worker-#32%failure.IoomFailureHandlerTest1%", id=340, > state=TIMED_WAITING, blockCnt=0, waitCnt=2] > [17:35:15]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@30364216, > ownerName=null, ownerId=-1] > [17:35:15]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:15]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4367) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2253) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:15]W: [org.apache.ignite:ignite-core] > [17:35:15]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-3-#279%failure.IoomFailureHandlerTest1%", > id=338, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:15]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:15]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:15]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:15]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:15]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@38bb9d7a > [17:35:15]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@1e8ab90f > [17:35:15]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@78d6447a > [17:35:15]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:15]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-2-#278%failure.IoomFailureHandlerTest1%", > id=337, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@7a729f84 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@1e469dfd > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@554f0dfb > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-1-#277%failure.IoomFailureHandlerTest1%", > id=336, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@1f7076bc > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@71904469 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@5b9396d3 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-0-#276%failure.IoomFailureHandlerTest1%", > id=335, state=RUNNABLE, blockCnt=4, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@761e788f > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@6a472566 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@1907874b > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="nio-acceptor-tcp-comm-#280%failure.IoomFailureHandlerTest1%", id=339, > state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.Util$3@5e65afb6 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@5b3a7ef5 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@8851ce1 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2922) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2870) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-timeout-worker-#275%failure.IoomFailureHandlerTest1%", id=334, > state=TIMED_WAITING, blockCnt=1, waitCnt=965] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.lang.Object@2d9dff65, ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="jvm-pause-detector-worker", id=327, state=TIMED_WAITING, blockCnt=0, > waitCnt=5964] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.LongJVMPauseDetector$1.run(LongJVMPauseDetector.java:100) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-7-#268%failure.IoomFailureHandlerTest1%", id=326, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-6-#267%failure.IoomFailureHandlerTest1%", id=325, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-5-#266%failure.IoomFailureHandlerTest1%", id=324, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-4-#265%failure.IoomFailureHandlerTest1%", id=323, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-3-#264%failure.IoomFailureHandlerTest1%", id=322, > state=WAITING, blockCnt=0, waitCnt=1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-2-#263%failure.IoomFailureHandlerTest1%", id=321, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-1-#262%failure.IoomFailureHandlerTest1%", id=320, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-0-#261%failure.IoomFailureHandlerTest1%", id=319, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-7-#260%failure.IoomFailureHandlerTest1%", id=318, > state=WAITING, blockCnt=0, waitCnt=1247] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-6-#259%failure.IoomFailureHandlerTest1%", id=317, > state=WAITING, blockCnt=122, waitCnt=2220] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-5-#258%failure.IoomFailureHandlerTest1%", id=316, > state=WAITING, blockCnt=0, waitCnt=1265] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-4-#257%failure.IoomFailureHandlerTest1%", id=315, > state=WAITING, blockCnt=0, waitCnt=1362] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-3-#256%failure.IoomFailureHandlerTest1%", id=314, > state=WAITING, blockCnt=0, waitCnt=1113] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-2-#255%failure.IoomFailureHandlerTest1%", id=313, > state=WAITING, blockCnt=0, waitCnt=1152] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-1-#254%failure.IoomFailureHandlerTest1%", id=312, > state=WAITING, blockCnt=0, waitCnt=1347] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-0-#253%failure.IoomFailureHandlerTest1%", id=311, > state=WAITING, blockCnt=0, waitCnt=1228] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="services-deployment-worker-#252%failure.IoomFailureHandlerTest0%", > id=309, state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5ef6fd7f, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.service.ServiceDeploymentManager$ServicesDeploymentWorker.body(ServiceDeploymentManager.java:461) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="exchange-worker-#251%failure.IoomFailureHandlerTest0%", id=308, > state=TIMED_WAITING, blockCnt=8, waitCnt=35] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5633ed82, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2809) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2735) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="disco-event-worker-#250%failure.IoomFailureHandlerTest0%", id=307, > state=WAITING, blockCnt=0, waitCnt=305] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@767191b1, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2884) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2854) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-ip-finder-cleaner-#31%failure.IoomFailureHandlerTest0%", > id=306, state=TIMED_WAITING, blockCnt=0, waitCnt=5] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$IpFinderCleaner.body(ServerImpl.java:1962) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-srvr-#30%failure.IoomFailureHandlerTest0%", id=305, > state=RUNNABLE, blockCnt=1, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.net.PlainSocketImpl.socketAccept(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.net.ServerSocket.implAccept(ServerSocket.java:545) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.net.ServerSocket.accept(ServerSocket.java:513) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5890) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5808) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-disco-msg-worker-#29%failure.IoomFailureHandlerTest0%", id=304, > state=TIMED_WAITING, blockCnt=0, waitCnt=30128] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@34d52ecd, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7228) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2715) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7164) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="disco-notifier-worker-#249%failure.IoomFailureHandlerTest0%", id=303, > state=WAITING, blockCnt=1, waitCnt=321] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5d21202d, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2689) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2734) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="ttl-cleanup-worker-#248%failure.IoomFailureHandlerTest0%", id=302, > state=TIMED_WAITING, blockCnt=0, waitCnt=600] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7625) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:153) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-data-loader-flusher-#245%failure.IoomFailureHandlerTest0%", > id=298, state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@14b789f6, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.DelayQueue.take(DelayQueue.java:211) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:109) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-client-listener-3-#242%failure.IoomFailureHandlerTest0%", > id=295, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@6eb17ec8 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@730f9695 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@277bf091 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-client-listener-2-#241%failure.IoomFailureHandlerTest0%", > id=294, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@fab35b1 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@6a094db2 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@65bdd558 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-client-listener-1-#240%failure.IoomFailureHandlerTest0%", > id=293, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@1dba4e06 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@4beaf6bd > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@6b2e46af > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-client-listener-0-#239%failure.IoomFailureHandlerTest0%", > id=292, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@60325987 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@2f37f1f9 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@3374b5bc > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="nio-acceptor-client-listener-#243%failure.IoomFailureHandlerTest0%", > id=296, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.Util$3@2af69643 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@587a1cfb > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@48528634 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2922) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2870) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-sharedfs-timeout-worker-#28%failure.IoomFailureHandlerTest0%", > id=291, state=TIMED_WAITING, blockCnt=0, waitCnt=61] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.lang.Object@5e1d03d7, ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.checkpoint.sharedfs.SharedFsTimeoutTask.body(SharedFsTimeoutTask.java:99) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="tcp-comm-worker-#27%failure.IoomFailureHandlerTest0%", id=290, > state=TIMED_WAITING, blockCnt=0, waitCnt=1] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4047d2d9, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4367) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2253) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:61) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-3-#237%failure.IoomFailureHandlerTest0%", > id=288, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@48d293ee > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@146dcfe6 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@22587507 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-2-#236%failure.IoomFailureHandlerTest0%", > id=287, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@5c77ba8f > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@660f0c > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@7a0ef219 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-1-#235%failure.IoomFailureHandlerTest0%", > id=286, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@1b1f5012 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@7add323c > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@4a734c04 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-nio-worker-tcp-comm-0-#234%failure.IoomFailureHandlerTest0%", > id=285, state=RUNNABLE, blockCnt=17, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.util.nio.SelectedSelectionKeySet@4760f169 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@261ea657 > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@35c12c7a > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2145) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1791) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="nio-acceptor-tcp-comm-#238%failure.IoomFailureHandlerTest0%", id=289, > state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.Util$3@658255aa > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > java.util.Collections$UnmodifiableSet@25290bca > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > sun.nio.ch.EPollSelectorImpl@76563d26 > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2922) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2870) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="grid-timeout-worker-#233%failure.IoomFailureHandlerTest0%", id=284, > state=TIMED_WAITING, blockCnt=1, waitCnt=1069] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.lang.Object@4ac86d6a, ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:269) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="jvm-pause-detector-worker", id=277, state=TIMED_WAITING, blockCnt=0, > waitCnt=5995] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.LongJVMPauseDetector$1.run(LongJVMPauseDetector.java:100) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-7-#226%failure.IoomFailureHandlerTest0%", id=276, > state=WAITING, blockCnt=0, waitCnt=1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-6-#225%failure.IoomFailureHandlerTest0%", id=275, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-5-#224%failure.IoomFailureHandlerTest0%", id=274, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-4-#223%failure.IoomFailureHandlerTest0%", id=273, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-3-#222%failure.IoomFailureHandlerTest0%", id=272, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-2-#221%failure.IoomFailureHandlerTest0%", id=271, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-1-#220%failure.IoomFailureHandlerTest0%", id=270, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="data-streamer-stripe-0-#219%failure.IoomFailureHandlerTest0%", id=269, > state=WAITING, blockCnt=0, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-7-#218%failure.IoomFailureHandlerTest0%", id=268, > state=WAITING, blockCnt=0, waitCnt=1276] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-6-#217%failure.IoomFailureHandlerTest0%", id=267, > state=WAITING, blockCnt=0, waitCnt=1253] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-5-#216%failure.IoomFailureHandlerTest0%", id=266, > state=WAITING, blockCnt=0, waitCnt=1231] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-4-#215%failure.IoomFailureHandlerTest0%", id=265, > state=WAITING, blockCnt=0, waitCnt=1206] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-3-#214%failure.IoomFailureHandlerTest0%", id=264, > state=WAITING, blockCnt=0, waitCnt=1272] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-2-#213%failure.IoomFailureHandlerTest0%", id=263, > state=WAITING, blockCnt=0, waitCnt=1253] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-1-#212%failure.IoomFailureHandlerTest0%", id=262, > state=WAITING, blockCnt=0, waitCnt=1268] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="sys-stripe-0-#211%failure.IoomFailureHandlerTest0%", id=261, > state=WAITING, blockCnt=0, waitCnt=1259] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:672) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:494) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="test-runner-#210%failure.IoomFailureHandlerTest%", id=260, > state=WAITING, blockCnt=11, waitCnt=10040] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.lang.Object@508a65bf, ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Object.wait(Object.java:502) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.worker.GridWorker.join(GridWorker.java:243) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.join(IgniteUtils.java:7643) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager.shutdownCheckpointer(GridCacheDatabaseSharedManager.java:1260) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager.onKernalStop0(GridCacheDatabaseSharedManager.java:991) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:120) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:1153) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.IgniteKernal.stop0(IgniteKernal.java:2324) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.IgniteKernal.stop(IgniteKernal.java:2272) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2614) > [17:35:16]W: [org.apache.ignite:ignite-core] - locked > o.a.i.i.IgnitionEx$IgniteNamedInstance@3d904e9c > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2577) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.IgnitionEx.stop(IgnitionEx.java:379) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.Ignition.stop(Ignition.java:223) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:1142) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:1123) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:1422) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.failure.IoomFailureHandlerTest.testIoomErrorHandling(IoomFailureHandlerTest.java:168) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.failure.IoomFailureHandlerTest.testIoomErrorPdsHandling(IoomFailureHandlerTest.java:113) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.reflect.Method.invoke(Method.java:498) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2088) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread > [name="ignite-clock-for-tests", id=16, state=TIMED_WAITING, blockCnt=0, > waitCnt=31079] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Thread.sleep(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.GridTestClockTimer.run(GridTestClockTimer.java:50) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread [name="resolver-3", > id=15, state=WAITING, blockCnt=1, waitCnt=1] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@17f2dd85, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread [name="resolver-2", > id=14, state=WAITING, blockCnt=1, waitCnt=2] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@17f2dd85, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread [name="resolver-1", > id=13, state=WAITING, blockCnt=0, waitCnt=1] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@17f2dd85, > ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Thread.run(Thread.java:748) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread [name="Signal > Dispatcher", id=4, state=RUNNABLE, blockCnt=0, waitCnt=0] > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread [name="Finalizer", id=3, > state=WAITING, blockCnt=26, waitCnt=10] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.lang.ref.ReferenceQueue$Lock@1e58512c, ownerName=null, > ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread [name="Reference > Handler", id=2, state=WAITING, blockCnt=7, waitCnt=6] > [17:35:16]W: [org.apache.ignite:ignite-core] Lock > [object=java.lang.ref.Reference$Lock@210308d5, ownerName=null, ownerId=-1] > [17:35:16]W: [org.apache.ignite:ignite-core] at java.lang.Object.wait(Native > Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.Object.wait(Object.java:502) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.ref.Reference.tryHandlePending(Reference.java:191) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Thread [name="main", id=1, > state=RUNNABLE, blockCnt=3, waitCnt=5] > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.management.ThreadImpl.dumpThreads0(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1369) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2116) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.testframework.junits.JUnit3TestLegacySupport.runTestCase(JUnit3TestLegacySupport.java:64) > [17:35:16]W: [org.apache.ignite:ignite-core] at > o.a.i.testframework.junits.GridAbstractTest$2.evaluate(GridAbstractTest.java:190) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.rules.RunRules.evaluate(RunRules.java:20) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner.run(ParentRunner.java:309) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.Suite.runChild(Suite.java:127) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.Suite.runChild(Suite.java:26) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.junit.runners.ParentRunner.run(ParentRunner.java:309) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:239) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:160) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.reflect.Method.invoke(Method.java:498) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:206) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:160) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:83) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.runSuitesInProcess(InPluginVMSurefireStarter.java:84) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1107) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:954) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:832) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.cli.MavenCli.execute(MavenCli.java:956) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.cli.MavenCli.doMain(MavenCli.java:290) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.apache.maven.cli.MavenCli.main(MavenCli.java:194) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > [17:35:16]W: [org.apache.ignite:ignite-core] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [17:35:16]W: [org.apache.ignite:ignite-core] at > java.lang.reflect.Method.invoke(Method.java:498) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > [17:35:16]W: [org.apache.ignite:ignite-core] at > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) > [17:35:16]W: [org.apache.ignite:ignite-core] > [17:35:16]W: [org.apache.ignite:ignite-core] Locked synchronizers: > [17:35:16]W: [org.apache.ignite:ignite-core] > java.util.concurrent.locks.ReentrantLock$NonfairSync@22a736d7 > > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)