Please see the log, it seems to be OutOfMemory from JVM on Windows Server 2016. How can we address this issue?
^-- System thread pool [active=0, idle=7, qSize=0] [08:35:20,163][INFO][grid-timeout-worker-#22%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%][IgniteKernal%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56] FreeList [name=default##FreeList, buckets=256, dataPages=2657, reusePages=0] [08:35:21,897][SEVERE][query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%][GridCacheDistributedQueryManager] <NonEquitiesYoY> Failed to run query [qry=GridCacheQueryInfo [loc=false, trans=null, rdc=null, qry=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=null, limit=0, transform=null, part=null, incMeta=false, pageSize=1024, timeout=0, incBackups=false, forceLocal=false, dedup=false, prj=null, keepBinary=true, subjId=7d026e80-e6c9-4ac0-8c93-3cb1f56b0054, taskHash=0, mvccSnapshot=null, dataPageScanEnabled=null], locFut=null, sndId=7d026e80-e6c9-4ac0-8c93-3cb1f56b0054, reqId=9, incMeta=false, all=false], node=10276c3c-a550-490e-bc48-edf9596dd1dd] java.lang.OutOfMemoryError: Java heap space at org.apache.ignite.internal.processors.cache.IncompleteCacheObject.<init>(IncompleteCacheObject.java:44) at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1344) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:625) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:445) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:356) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:261) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:160) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:131) at org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55) at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:130) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:419) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5787) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5553) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.init(BPlusTree.java:5479) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findLowerUnbounded(BPlusTree.java:1048) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1098) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:164) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1080) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2929) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2899) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2893) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2881) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$3.onHasNext(IgniteCacheOffheapManagerImpl.java:942) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.advance(GridCacheQueryManager.java:3062) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.onHasNext(GridCacheQueryManager.java:3028) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1314) [08:35:21,897][SEVERE][query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%][GridCacheDistributedQueryManager] <NonEquitiesYoY> Failed to run query. java.lang.OutOfMemoryError: Java heap space at org.apache.ignite.internal.processors.cache.IncompleteCacheObject.<init>(IncompleteCacheObject.java:44) at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1344) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:625) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:445) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:356) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:261) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:160) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:131) at org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55) at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:130) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:419) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5787) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5553) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.init(BPlusTree.java:5479) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findLowerUnbounded(BPlusTree.java:1048) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1098) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:164) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1080) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2929) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2899) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2893) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2881) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$3.onHasNext(IgniteCacheOffheapManagerImpl.java:942) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.advance(GridCacheQueryManager.java:3062) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.onHasNext(GridCacheQueryManager.java:3028) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1314) [08:35:21,897][SEVERE][query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%][GridCacheIoManager] Failed processing message [senderId=7d026e80-e6c9-4ac0-8c93-3cb1f56b0054, msg=GridCacheQueryRequest [id=9, cacheName=NonEquitiesYoY, type=SCAN, fields=false, clause=null, limit=0, clsName=null, keyValFilter=null, rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false, incMeta=false, all=false, keepBinary=true, subjId=7d026e80-e6c9-4ac0-8c93-3cb1f56b0054, taskHash=0, part=-1, topVer=AffinityTopologyVersion [topVer=163, minorTopVer=0], mvccSnapshot=null, flags=0, super=GridCacheIdMessage [cacheId=-1867634189, super=GridCacheMessage [msgId=10, depInfo=null, lastAffChangedTopVer=AffinityTopologyVersion [topVer=158, minorTopVer=1], err=null, skipPrepare=false]]]] java.lang.OutOfMemoryError: Java heap space at org.apache.ignite.internal.processors.cache.IncompleteCacheObject.<init>(IncompleteCacheObject.java:44) at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1344) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:625) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:445) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:356) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:261) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:160) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:131) at org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55) at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:130) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:419) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5787) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5553) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.init(BPlusTree.java:5479) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findLowerUnbounded(BPlusTree.java:1048) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1098) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:164) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1080) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2929) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2899) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2893) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2881) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$3.onHasNext(IgniteCacheOffheapManagerImpl.java:942) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.advance(GridCacheQueryManager.java:3062) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.onHasNext(GridCacheQueryManager.java:3028) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1314) [08:35:22,163][SEVERE][query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: Java heap space]] java.lang.OutOfMemoryError: Java heap space at org.apache.ignite.internal.processors.cache.IncompleteCacheObject.<init>(IncompleteCacheObject.java:44) at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1344) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:625) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:445) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:356) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:261) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:160) at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:131) at org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55) at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:130) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:419) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5787) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5553) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.init(BPlusTree.java:5479) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findLowerUnbounded(BPlusTree.java:1048) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1098) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:164) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1080) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2929) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2899) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2893) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2881) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$3.onHasNext(IgniteCacheOffheapManagerImpl.java:942) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.advance(GridCacheQueryManager.java:3062) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$ScanQueryIterator.onHasNext(GridCacheQueryManager.java:3028) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1314) [08:35:22,179][WARNING][query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%][CacheDiagnosticManager] Page locks dump: Thread=[name=exchange-worker-#48%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=66], state=TIMED_WAITING Locked pages = [] Locked pages log: name=exchange-worker-#48%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=7400], state=RUNNABLE Locked pages = [] Locked pages log: name=query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-0-#1%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=11], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-0-#1%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-1-#2%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=12], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-1-#2%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-2-#3%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=13], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-2-#3%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-3-#4%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=14], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-3-#4%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-4-#5%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=15], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-4-#5%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-5-#6%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=16], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-5-#6%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-6-#7%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=17], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-6-#7%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) Thread=[name=sys-stripe-7-#8%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%, id=18], state=WAITING Locked pages = [] Locked pages log: name=sys-stripe-7-#8%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56% time=(1615394122179, 2021-03-10 08:35:22.179) [08:35:22,179][SEVERE][query-#7335%ignite-instance-9329856b-236a-4482-a29e-d9e7ace51c56%][] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: Java heap space]] From: Stephen Darlington <stephen.darling...@gridgain.com> Sent: Wednesday, March 10, 2021 9:35 AM To: user@ignite.apache.org Subject: Re: Cluster keeps crashing with SEVERE message This is an EXTERNAL EMAIL. Stop and think before clicking a link or opening attachments. We’d need to see more of the stack trace and see your configuration before this could be diagnosed. It looks like a node abruptly left the cluster. If you’re losing data it seems like you’ve not configured any backups. On 10 Mar 2021, at 16:53, Josh Katz <josh.katz.contrac...@dodgeandcox.com<mailto:josh.katz.contrac...@dodgeandcox.com>> wrote: After we load 35 million records from the database into the cluster. When we query it using a scan query we get the following SEVERE error, After that the node are restarting and the caches all show 0 keys and we have to restart the initial load. Is there a way to salvage the data so we don’t have to reload every time? For example, if we add persistence at this stage the initial load will be slower correct? How can we avoid seeing this error? “[SEVERE] Failed to process selector key” <image001.png> Thanks, Josh ________________________________ Please follow the hyperlink to important disclosures. https://www.dodgeandcox.com/disclosures/email_disclosure_funds.html