[jira] [Updated] (IGNITE-8742) Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: WAL manager failure does not stoped execution
[ https://issues.apache.org/jira/browse/IGNITE-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-8742: --- Fix Version/s: 2.7 > Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: > WAL manager failure does not stoped execution > --- > > Key: IGNITE-8742 > URL: https://issues.apache.org/jira/browse/IGNITE-8742 > Project: Ignite > Issue Type: Test > Components: persistence >Reporter: Dmitriy Pavlov >Assignee: Dmitriy Sorokin >Priority: Major > Labels: MakeTeamcityGreenAgain > Fix For: 2.7 > > > https://ci.ignite.apache.org/viewLog.html?buildId=1366882&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_PdsDirectIo2 > Test > org.apache.ignite.internal.processors.cache.persistence.IgniteNativeIoWalFlushFsyncSelfTest#testFailAfterStart > emulates problem with disc space using exception. > In direct IO environment real IO with disk is performed, tmpfs is not used. > Sometimes this error can come from rollover() of segment, failure handler > reacted accordingly. > {noformat} > detected. Will be handled accordingly to configured handler [hnd=class > o.a.i.failure.StopNodeFailureHandler, failureCtx=FailureContext > [type=CRITICAL_ERROR, err=class o.a.i.i.pagemem.wal.StorageException: Unable > to write]] > class org.apache.ignite.internal.pagemem.wal.StorageException: Unable to write > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.writeBuffer(FsyncModeFileWriteAheadLogManager.java:2964) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2640) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2572) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2525) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.close(FsyncModeFileWriteAheadLogManager.java:2795) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$700(FsyncModeFileWriteAheadLogManager.java:2340) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.rollOver(FsyncModeFileWriteAheadLogManager.java:1029) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.log(FsyncModeFileWriteAheadLogManager.java:673) > {noformat} > But test seems to be not able to stop, node stopper thread tries to stop > cache, flush WAL. flush wait for rollover, which will never happen. > {noformat} > Thread [name="node-stopper", id=2836, state=WAITING, blockCnt=7, waitCnt=9] > Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@47f6473, > ownerName=null, ownerId=-1] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976) > at o.a.i.i.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7473) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2546) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.fsync(FsyncModeFileWriteAheadLogManager.java:2750) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$2000(FsyncModeFileWriteAheadLogManager.java:2340) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.flush(FsyncModeFileWriteAheadLogManager.java:699) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1243) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCaches(GridCacheProcessor.java:969) > at > o.a.i.i.processors.cache.GridCacheProcessor.stop(GridCacheProcessor.java:943) > at o.a.i.i.IgniteKernal.stop0(IgniteKernal.java:2289) > at o.a.i.i.IgniteKernal.stop(IgniteKernal.java:2167) > at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588) > - locked o.a.i.i.IgnitionEx$IgniteNamedInstance@90f6bfd > at o.a.i.i.IgnitionEx
[jira] [Updated] (IGNITE-8742) Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: WAL manager failure does not stoped execution
[ https://issues.apache.org/jira/browse/IGNITE-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Pavlov updated IGNITE-8742: --- Labels: MakeTeamcityGreenAgain (was: ) > Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: > WAL manager failure does not stoped execution > --- > > Key: IGNITE-8742 > URL: https://issues.apache.org/jira/browse/IGNITE-8742 > Project: Ignite > Issue Type: Test > Components: persistence >Reporter: Dmitriy Pavlov >Priority: Major > Labels: MakeTeamcityGreenAgain > > https://ci.ignite.apache.org/viewLog.html?buildId=1366882&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_PdsDirectIo2 > Test > org.apache.ignite.internal.processors.cache.persistence.IgniteNativeIoWalFlushFsyncSelfTest#testFailAfterStart > emulates problem with disc space using exception. > In direct IO environment real IO with disk is performed, tmpfs is not used. > Sometimes this error can come from rollover() of segment, failure handler > reacted accordingly. > {noformat} > detected. Will be handled accordingly to configured handler [hnd=class > o.a.i.failure.StopNodeFailureHandler, failureCtx=FailureContext > [type=CRITICAL_ERROR, err=class o.a.i.i.pagemem.wal.StorageException: Unable > to write]] > class org.apache.ignite.internal.pagemem.wal.StorageException: Unable to write > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.writeBuffer(FsyncModeFileWriteAheadLogManager.java:2964) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2640) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2572) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2525) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.close(FsyncModeFileWriteAheadLogManager.java:2795) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$700(FsyncModeFileWriteAheadLogManager.java:2340) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.rollOver(FsyncModeFileWriteAheadLogManager.java:1029) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.log(FsyncModeFileWriteAheadLogManager.java:673) > {noformat} > But test seems to be not able to stop, node stopper thread tries to stop > cache, flush WAL. flush wait for rollover, which will never happen. > {noformat} > Thread [name="node-stopper", id=2836, state=WAITING, blockCnt=7, waitCnt=9] > Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@47f6473, > ownerName=null, ownerId=-1] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976) > at o.a.i.i.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7473) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2546) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.fsync(FsyncModeFileWriteAheadLogManager.java:2750) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$2000(FsyncModeFileWriteAheadLogManager.java:2340) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.flush(FsyncModeFileWriteAheadLogManager.java:699) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1243) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCaches(GridCacheProcessor.java:969) > at > o.a.i.i.processors.cache.GridCacheProcessor.stop(GridCacheProcessor.java:943) > at o.a.i.i.IgniteKernal.stop0(IgniteKernal.java:2289) > at o.a.i.i.IgniteKernal.stop(IgniteKernal.java:2167) > at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588) > - locked o.a.i.i.IgnitionEx$IgniteNamedInstance@90f6bfd > at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.ja
[jira] [Updated] (IGNITE-8742) Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: WAL manager failure does not stoped execution
[ https://issues.apache.org/jira/browse/IGNITE-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Pavlov updated IGNITE-8742: --- Description: https://ci.ignite.apache.org/viewLog.html?buildId=1366882&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_PdsDirectIo2 Test org.apache.ignite.internal.processors.cache.persistence.IgniteNativeIoWalFlushFsyncSelfTest#testFailAfterStart emulates problem with disc space using exception. In direct IO environment real IO with disk is performed, tmpfs is not used. Sometimes this error can come from rollover() of segment, failure handler reacted accordingly. {noformat} detected. Will be handled accordingly to configured handler [hnd=class o.a.i.failure.StopNodeFailureHandler, failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.pagemem.wal.StorageException: Unable to write]] class org.apache.ignite.internal.pagemem.wal.StorageException: Unable to write at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.writeBuffer(FsyncModeFileWriteAheadLogManager.java:2964) at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2640) at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2572) at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2525) at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.close(FsyncModeFileWriteAheadLogManager.java:2795) at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$700(FsyncModeFileWriteAheadLogManager.java:2340) at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.rollOver(FsyncModeFileWriteAheadLogManager.java:1029) at org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.log(FsyncModeFileWriteAheadLogManager.java:673) {noformat} But test seems to be not able to stop, node stopper thread tries to stop cache, flush WAL. flush wait for rollover, which will never happen. {noformat} Thread [name="node-stopper", id=2836, state=WAITING, blockCnt=7, waitCnt=9] Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@47f6473, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976) at o.a.i.i.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7473) at o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2546) at o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.fsync(FsyncModeFileWriteAheadLogManager.java:2750) at o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$2000(FsyncModeFileWriteAheadLogManager.java:2340) at o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.flush(FsyncModeFileWriteAheadLogManager.java:699) at o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1243) at o.a.i.i.processors.cache.GridCacheProcessor.stopCaches(GridCacheProcessor.java:969) at o.a.i.i.processors.cache.GridCacheProcessor.stop(GridCacheProcessor.java:943) at o.a.i.i.IgniteKernal.stop0(IgniteKernal.java:2289) at o.a.i.i.IgniteKernal.stop(IgniteKernal.java:2167) at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588) - locked o.a.i.i.IgnitionEx$IgniteNamedInstance@90f6bfd at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2551) at o.a.i.i.IgnitionEx.stop(IgnitionEx.java:372) at o.a.i.failure.StopNodeFailureHandler$1.run(StopNodeFailureHandler.java:36) at java.lang.Thread.run(Thread.java:748) {noformat} It seems invalidating environment of WAL manager is not working propertly. was: Test org.apache.ignite.internal.processors.cache.persistence.IgniteNativeIoWalFlushFsyncSelfTest#testFailAfterStart emulates problem with disc space using exception. In direct IO environment real IO with disk is performed, tmpfs is not used. Sometimes this error can come from rollover() of segment, failure handler reacted accordingly. {noformat} detected. Will be handled accordingly to configured h
[jira] [Updated] (IGNITE-8742) Direct IO 2 suite is timed out by out of disk space failure emulation test: WAL manager failure does not stoped execution
[ https://issues.apache.org/jira/browse/IGNITE-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Pavlov updated IGNITE-8742: --- Summary: Direct IO 2 suite is timed out by out of disk space failure emulation test: WAL manager failure does not stoped execution (was: Direct IO 2 suite is timed out by out of disk space failure emulation test: WAL manager failure does not stoped.) > Direct IO 2 suite is timed out by out of disk space failure emulation test: > WAL manager failure does not stoped execution > - > > Key: IGNITE-8742 > URL: https://issues.apache.org/jira/browse/IGNITE-8742 > Project: Ignite > Issue Type: Test > Components: persistence >Reporter: Dmitriy Pavlov >Priority: Major > > Test > org.apache.ignite.internal.processors.cache.persistence.IgniteNativeIoWalFlushFsyncSelfTest#testFailAfterStart > emulates problem with disc space using exception. > In direct IO environment real IO with disk is performed, tmpfs is not used. > Sometimes this error can come from rollover() of segment, failure handler > reacted accordingly. > {noformat} > detected. Will be handled accordingly to configured handler [hnd=class > o.a.i.failure.StopNodeFailureHandler, failureCtx=FailureContext > [type=CRITICAL_ERROR, err=class o.a.i.i.pagemem.wal.StorageException: Unable > to write]] > class org.apache.ignite.internal.pagemem.wal.StorageException: Unable to write > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.writeBuffer(FsyncModeFileWriteAheadLogManager.java:2964) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2640) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2572) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2525) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.close(FsyncModeFileWriteAheadLogManager.java:2795) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$700(FsyncModeFileWriteAheadLogManager.java:2340) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.rollOver(FsyncModeFileWriteAheadLogManager.java:1029) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.log(FsyncModeFileWriteAheadLogManager.java:673) > {noformat} > But test seems to be not able to stop, node stopper thread tries to stop > cache, flush WAL. flush wait for rollover, which will never happen. > {noformat} > Thread [name="node-stopper", id=2836, state=WAITING, blockCnt=7, waitCnt=9] > Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@47f6473, > ownerName=null, ownerId=-1] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976) > at o.a.i.i.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7473) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2546) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.fsync(FsyncModeFileWriteAheadLogManager.java:2750) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$2000(FsyncModeFileWriteAheadLogManager.java:2340) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.flush(FsyncModeFileWriteAheadLogManager.java:699) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1243) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCaches(GridCacheProcessor.java:969) > at > o.a.i.i.processors.cache.GridCacheProcessor.stop(GridCacheProcessor.java:943) > at o.a.i.i.IgniteKernal.stop0(IgniteKernal.java:2289) > at o.a.i.i.IgniteKernal.stop(IgniteKernal.java:2167) > at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588) > - locked o.a.i.i.IgnitionEx$IgniteNamedInstance@90f6bfd > at o.a.i.i.Ignition
[jira] [Updated] (IGNITE-8742) Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: WAL manager failure does not stoped execution
[ https://issues.apache.org/jira/browse/IGNITE-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Pavlov updated IGNITE-8742: --- Summary: Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: WAL manager failure does not stoped execution (was: Direct IO 2 suite is timed out by out of disk space failure emulation test: WAL manager failure does not stoped execution) > Direct IO 2 suite is timed out by 'out of disk space' failure emulation test: > WAL manager failure does not stoped execution > --- > > Key: IGNITE-8742 > URL: https://issues.apache.org/jira/browse/IGNITE-8742 > Project: Ignite > Issue Type: Test > Components: persistence >Reporter: Dmitriy Pavlov >Priority: Major > > Test > org.apache.ignite.internal.processors.cache.persistence.IgniteNativeIoWalFlushFsyncSelfTest#testFailAfterStart > emulates problem with disc space using exception. > In direct IO environment real IO with disk is performed, tmpfs is not used. > Sometimes this error can come from rollover() of segment, failure handler > reacted accordingly. > {noformat} > detected. Will be handled accordingly to configured handler [hnd=class > o.a.i.failure.StopNodeFailureHandler, failureCtx=FailureContext > [type=CRITICAL_ERROR, err=class o.a.i.i.pagemem.wal.StorageException: Unable > to write]] > class org.apache.ignite.internal.pagemem.wal.StorageException: Unable to write > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.writeBuffer(FsyncModeFileWriteAheadLogManager.java:2964) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2640) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2572) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2525) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.close(FsyncModeFileWriteAheadLogManager.java:2795) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$700(FsyncModeFileWriteAheadLogManager.java:2340) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.rollOver(FsyncModeFileWriteAheadLogManager.java:1029) > at > org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.log(FsyncModeFileWriteAheadLogManager.java:673) > {noformat} > But test seems to be not able to stop, node stopper thread tries to stop > cache, flush WAL. flush wait for rollover, which will never happen. > {noformat} > Thread [name="node-stopper", id=2836, state=WAITING, blockCnt=7, waitCnt=9] > Lock > [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@47f6473, > ownerName=null, ownerId=-1] > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976) > at o.a.i.i.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7473) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2546) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.fsync(FsyncModeFileWriteAheadLogManager.java:2750) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$2000(FsyncModeFileWriteAheadLogManager.java:2340) > at > o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.flush(FsyncModeFileWriteAheadLogManager.java:699) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1243) > at > o.a.i.i.processors.cache.GridCacheProcessor.stopCaches(GridCacheProcessor.java:969) > at > o.a.i.i.processors.cache.GridCacheProcessor.stop(GridCacheProcessor.java:943) > at o.a.i.i.IgniteKernal.stop0(IgniteKernal.java:2289) > at o.a.i.i.IgniteKernal.stop(IgniteKernal.java:2167) > at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588) > - locked o.a.i.i.IgnitionEx$IgniteNamedInstance@90f6bfd > at o