Dmitriy Pavlov created IGNITE-8742:
--------------------------------------
Summary: Direct IO 2 suite is timed out by out of disk space
failure emulation test: WAL manager failure does not stoped.
Key: IGNITE-8742
URL: https://issues.apache.org/jira/browse/IGNITE-8742
Project: Ignite
Issue Type: Test
Components: persistence
Reporter: Dmitriy Pavlov
Test
org.apache.ignite.internal.processors.cache.persistence.IgniteNativeIoWalFlushFsyncSelfTest#testFailAfterStart
emulates problem with disc space using exception.
In direct IO environment real IO with disk is performed, tmpfs is not used.
Sometimes this error can come from rollover() of segment, failure handler
reacted accordingly.
{noformat}
detected. Will be handled accordingly to configured handler [hnd=class
o.a.i.failure.StopNodeFailureHandler, failureCtx=FailureContext
[type=CRITICAL_ERROR, err=class o.a.i.i.pagemem.wal.StorageException: Unable to
write]]
class org.apache.ignite.internal.pagemem.wal.StorageException: Unable to write
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.writeBuffer(FsyncModeFileWriteAheadLogManager.java:2964)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2640)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flush(FsyncModeFileWriteAheadLogManager.java:2572)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2525)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.close(FsyncModeFileWriteAheadLogManager.java:2795)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$700(FsyncModeFileWriteAheadLogManager.java:2340)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.rollOver(FsyncModeFileWriteAheadLogManager.java:1029)
at
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.log(FsyncModeFileWriteAheadLogManager.java:673)
{noformat}
But test seems to be not able to stop, node stopper thread tries to stop cache,
flush WAL. flush wait for rollover, which will never happen.
{noformat}
Thread [name="node-stopper", id=2836, state=WAITING, blockCnt=7, waitCnt=9]
Lock
[object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@47f6473,
ownerName=null, ownerId=-1]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitUninterruptibly(AbstractQueuedSynchronizer.java:1976)
at o.a.i.i.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7473)
at
o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.flushOrWait(FsyncModeFileWriteAheadLogManager.java:2546)
at
o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.fsync(FsyncModeFileWriteAheadLogManager.java:2750)
at
o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileWriteHandle.access$2000(FsyncModeFileWriteAheadLogManager.java:2340)
at
o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager.flush(FsyncModeFileWriteAheadLogManager.java:699)
at
o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1243)
at
o.a.i.i.processors.cache.GridCacheProcessor.stopCaches(GridCacheProcessor.java:969)
at
o.a.i.i.processors.cache.GridCacheProcessor.stop(GridCacheProcessor.java:943)
at o.a.i.i.IgniteKernal.stop0(IgniteKernal.java:2289)
at o.a.i.i.IgniteKernal.stop(IgniteKernal.java:2167)
at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588)
- locked o.a.i.i.IgnitionEx$IgniteNamedInstance@90f6bfd
at o.a.i.i.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2551)
at o.a.i.i.IgnitionEx.stop(IgnitionEx.java:372)
at
o.a.i.failure.StopNodeFailureHandler$1.run(StopNodeFailureHandler.java:36)
at java.lang.Thread.run(Thread.java:748)
{noformat}
It seems invalidating environment of WAL manager is not working propertly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
