Hello, I have a 2 server nodes and I configure only one node Data region
eviction policy and restart.
and I got a the error
[m
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
B+Tree is corrupted [pages(groupId, pageId)=[], msg=Runtime failure on
bounds: [lower=null, upper=null]]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5927)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1054)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:164)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.find(CacheDataTree.java:63)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1021)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cursor(IgniteCacheOffheapManagerImpl.java:2844)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.reservedIterator(IgniteCacheOffheapManagerImpl.java:1141)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.rebalanceIterator(IgniteCacheOffheapManagerImpl.java:1184)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:270)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.lambda$handleDemandMessage$1(GridDhtPreloader.java:370)
~[ignite-core-2.8.0.jar:2.8.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_181]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by:
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTreeRuntimeException:
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTreeRuntimeException:
java.lang.IllegalArgumentException: Invalid object type: 0
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findLowerUnbounded(BPlusTree.java:1000)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1039)
~[ignite-core-2.8.0.jar:2.8.0]
... 11 more
Caused by:
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTreeRuntimeException:
java.lang.IllegalArgumentException: Invalid object type: 0
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:290)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:160)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:131)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:130)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:417)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5608)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5376)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.init(BPlusTree.java:5302)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findLowerUnbounded(BPlusTree.java:989)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1039)
~[ignite-core-2.8.0.jar:2.8.0]
... 11 more
Caused by: java.lang.IllegalArgumentException: Invalid object type: 0
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toKeyCacheObject(CacheObjectBinaryProcessorImpl.java:1167)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFullRow(CacheDataRowAdapter.java:508)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:334)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:261)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:160)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:131)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:130)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:417)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5608)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5376)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.init(BPlusTree.java:5302)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findLowerUnbounded(BPlusTree.java:989)
~[ignite-core-2.8.0.jar:2.8.0]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1039)
~[ignite-core-2.8.0.jar:2.8.0]
... 11 more
[1;31m[2020.10.26 17:48:35.306] [ERROR] [o.a.i.i.p.f.FailureProcessor] A
critical problem with persistence data structures was detected. Please make
backup of persistence storage and WAL files for further analysis.
Persistence storage path: null WAL path: db/wal WAL archive path:
db/wal/archive
[m[33m[2020.10.26 17:48:35.311] [WARN ]
[o.a.i.i.p.c.CacheDiagnosticManager] Page locks dump:
Thread=[name=client-connector-#101%NexusCache1-1-2%, id=155], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#101%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#102%NexusCache1-1-2%, id=156], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#102%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#103%NexusCache1-1-2%, id=157], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#103%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#104%NexusCache1-1-2%, id=158], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#104%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#105%NexusCache1-1-2%, id=159], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#105%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#106%NexusCache1-1-2%, id=160], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#106%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#107%NexusCache1-1-2%, id=161], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#107%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#108%NexusCache1-1-2%, id=162], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#108%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#109%NexusCache1-1-2%, id=163], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#109%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#110%NexusCache1-1-2%, id=164], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#110%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#111%NexusCache1-1-2%, id=165], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#111%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#95%NexusCache1-1-2%, id=149], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#95%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#96%NexusCache1-1-2%, id=150], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#96%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#97%NexusCache1-1-2%, id=151], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#97%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#98%NexusCache1-1-2%, id=152], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#98%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=client-connector-#99%NexusCache1-1-2%, id=153], state=WAITING
Locked pages = []
Locked pages log: name=client-connector-#99%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=rebalance-striped-#138%NexusCache1-1-2%, id=200],
state=RUNNABLE
Locked pages = []
Locked pages log: name=rebalance-striped-#138%NexusCache1-1-2%
time=(1603702115306, 2020-10-26 17:48:35.306)
Thread=[name=sys-stripe-0-#1%NexusCache1-1-2%, id=22], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-0-#1%NexusCache1-1-2% time=(1603702115307,
2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-10-#11%NexusCache1-1-2%, id=32], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-10-#11%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-11-#12%NexusCache1-1-2%, id=33], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-11-#12%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-12-#13%NexusCache1-1-2%, id=34], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-12-#13%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-13-#14%NexusCache1-1-2%, id=35], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-13-#14%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-15-#16%NexusCache1-1-2%, id=37], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-15-#16%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-2-#3%NexusCache1-1-2%, id=24], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-2-#3%NexusCache1-1-2% time=(1603702115307,
2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-4-#5%NexusCache1-1-2%, id=26], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-4-#5%NexusCache1-1-2% time=(1603702115307,
2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-5-#6%NexusCache1-1-2%, id=27], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-5-#6%NexusCache1-1-2% time=(1603702115307,
2020-10-26 17:48:35.307)
Thread=[name=sys-stripe-7-#8%NexusCache1-1-2%, id=29], state=WAITING
Locked pages = []
Locked pages log: name=sys-stripe-7-#8%NexusCache1-1-2% time=(1603702115307,
2020-10-26 17:48:35.307)
Thread=[name=ttl-cleanup-worker-#68%NexusCache1-1-2%, id=120],
state=TIMED_WAITING
Locked pages = []
Locked pages log: name=ttl-cleanup-worker-#68%NexusCache1-1-2%
time=(1603702115307, 2020-10-26 17:48:35.307)
[m[32m[2020.10.26 17:48:35.313] [INFO ] [o.a.i.i.p.d.DiagnosticProcessor]
Operation was started: operation = DiagnosticPageHistory
[m[32m[2020.10.26 17:48:35.313] [INFO ]
[o.a.i.i.p.d.PageHistoryDiagnoster] Skipping dump page history due to WAL
not configured
[m[32m[2020.10.26 17:48:35.313] [INFO ] [o.a.i.i.p.d.DiagnosticProcessor]
Operation was success: operation = DiagnosticPageHistory, elapsedTime = 0 ms
[m[1;31m[2020.10.26 17:48:35.313] [ERROR] [] JVM will be halted
immediately due to the failure: [failureCtx=FailureContext
[type=CRITICAL_ERROR, err=class
o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is
corrupted [pages(groupId, pageId)=[], msg=Runtime failure on bounds:
[lower=null, upper=null]]]]
[m[33m[2020.10.26 17:56:47.677] [WARN ] [o.a.i.i.u.t.G] Ignite work
directory is not provided, automatically resolved to:
/ERP/Domains/NexusCache/work
[m[32m[2020.10.26 17:56:47.782] [INFO ]
[o.a.i.i.IgniteKernal%NexusCache1-1-2]
and the other node was shut down and It was start okay after 2nodes are shut
down and start all.
I don't use persistence mode for region.
What is the reason of the error?
I'll wait for reply.
Thank you so much.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
