Oleg Valuyskiy created IGNITE-21770: ---------------------------------------
Summary: Improve logging for the ‘Probably disk is too busy, please check your device’ event (when reading from disk) Key: IGNITE-21770 URL: https://issues.apache.org/jira/browse/IGNITE-21770 Project: Ignite Issue Type: Improvement Reporter: Oleg Valuyskiy Assignee: Oleg Valuyskiy Make logging messages more specific and include more information that could be necessary for analysis. Stack trace example: {code:java} 2024-03-15 04:23:25.395 [ERROR][ForkJoinPool.commonPool-worker-41][] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.processors.cache.persistence.StorageException: Failed to read page [file=/opt/ignite/ssd/data/cell_6_node_4/cacheGroup-part/part-15909.bin, pageId=349803611487542]]] org.apache.ignite.internal.processors.cache.persistence.StorageException: Failed to read page [file=/opt/ignite/ssd/data/cell_6_node_4/cacheGroup-part/part-15909.bin, pageId=349803611487542] at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:547) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:487) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageReadWriteManagerImpl.read(PageReadWriteManagerImpl.java:69) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:522) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:930) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:741) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:730) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:304) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:424) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:6168) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5917) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5971) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:6221) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$3.onHasNext(IgniteCacheOffheapManagerImpl.java:913) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:56) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.verify.IdleVerifyUtility.calculatePartitionHash(IdleVerifyUtility.java:269) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.verify.VerifyBackupPartitionsTaskV2$VerifyBackupPartitionsJobV2.lambda$calculatePartitionHashAsync$3(VerifyBackupPartitionsTaskV2.java:478) ~[ignite-core-14.1.2.jar:14.1.2] at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448) ~[?:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ~[?:?] at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[?:?] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ~[?:?] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) ~[?:?] Caused by: java.io.IOException: Probably disk is too busy, please check your device. at org.apache.ignite.internal.processors.cache.persistence.file.AbstractFileIO.fully(AbstractFileIO.java:66) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.file.AbstractFileIO.readFully(AbstractFileIO.java:88) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.readWithFailover(FilePageStore.java:897) ~[ignite-core-14.1.2.jar:14.1.2] at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:512) ~[ignite-core-14.1.2.jar:14.1.2] ... 27 more {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)