[
https://issues.apache.org/jira/browse/IGNITE-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16815412#comment-16815412
]
Denis Chudov edited comment on IGNITE-8876 at 4/11/19 1:25 PM:
---------------------------------------------------------------
I tried to reproduce this error by deleting cache concurrently while
destroyEvictedPartitions is in progress. This scenario is not possible now
because destroyCache awaits the end of checkpoint operation.
{code:java}
Thread
[name="sys-#99%persistence.CheckpointDestroyEvictedPartitionsAssertionTest1%",
id=136, state=WAITING, blockCnt=0, waitCnt=11]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
o.a.i.i.processors.cache.GridCacheProcessor.removeOffheapListenerAfterCheckpoint(GridCacheProcessor.java:3131)
at
o.a.i.i.processors.cache.GridCacheProcessor.processCacheStopRequestOnExchangeDone(GridCacheProcessor.java:3044)
at
o.a.i.i.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:3187)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2203)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processFullMessage(GridDhtPartitionsExchangeFuture.java:4174)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$1600(GridDhtPartitionsExchangeFuture.java:146)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:3867)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:3855)
at
o.a.i.i.util.future.GridFutureAdapter.notifyListener$$$capture(GridFutureAdapter.java:399)
at o.a.i.i.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java)
at o.a.i.i.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveFullMessage(GridDhtPartitionsExchangeFuture.java:3855)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager.processFullPartitionUpdate(GridCachePartitionExchangeManager.java:1770)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:412)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:399)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:3406)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:3385)
at
o.a.i.i.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1126)
at
o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
at
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
at
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
at
o.a.i.i.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
at
o.a.i.i.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
at
o.a.i.i.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1561)
at
o.a.i.i.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1189)
at
o.a.i.i.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at o.a.i.i.managers.communication.GridIoManager$8.run(GridIoManager.java:1086)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
was (Author: denis chudov):
I tried to reproduce this error by deleting concurrently cache while
destroyEvictedPartitions is in progress. This scenario is not possible now
because destroyCache awaits the end of checkpoint operation.
{code:java}
Thread
[name="sys-#99%persistence.CheckpointDestroyEvictedPartitionsAssertionTest1%",
id=136, state=WAITING, blockCnt=0, waitCnt=11]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
o.a.i.i.processors.cache.GridCacheProcessor.removeOffheapListenerAfterCheckpoint(GridCacheProcessor.java:3131)
at
o.a.i.i.processors.cache.GridCacheProcessor.processCacheStopRequestOnExchangeDone(GridCacheProcessor.java:3044)
at
o.a.i.i.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:3187)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2203)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processFullMessage(GridDhtPartitionsExchangeFuture.java:4174)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$1600(GridDhtPartitionsExchangeFuture.java:146)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:3867)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:3855)
at
o.a.i.i.util.future.GridFutureAdapter.notifyListener$$$capture(GridFutureAdapter.java:399)
at o.a.i.i.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java)
at o.a.i.i.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
at
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveFullMessage(GridDhtPartitionsExchangeFuture.java:3855)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager.processFullPartitionUpdate(GridCachePartitionExchangeManager.java:1770)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:412)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:399)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:3406)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:3385)
at
o.a.i.i.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1126)
at
o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
at
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
at
o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
at
o.a.i.i.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
at
o.a.i.i.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
at
o.a.i.i.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1561)
at
o.a.i.i.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1189)
at
o.a.i.i.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at o.a.i.i.managers.communication.GridIoManager$8.run(GridIoManager.java:1086)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
> Deactivate before checkpoint may lead to assertion and node failture.
> ---------------------------------------------------------------------
>
> Key: IGNITE-8876
> URL: https://issues.apache.org/jira/browse/IGNITE-8876
> Project: Ignite
> Issue Type: Bug
> Reporter: Alexei Scherbakov
> Assignee: Denis Chudov
> Priority: Major
> Fix For: 2.8
>
>
> {noformat}
> 2018-06-10 17:42:34.453 [INFO
> ][db-checkpoint-thread-#164%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint started [checkpointId=04d24209-ceaf-4c05-bcaa-bfebc8c83148,
> startPtr=FileWALPointer [idx=690, fileOff=656836779, len=41],
> checkpointLockWait=0ms, checkpointLockHoldTime=0ms,
> walCpRecordFsyncDuration=0ms, pages=80236, reason='partition destroy']
> 2018-06-10 17:42:34.470
> [ERROR][db-checkpoint-thread-#164%DPL_GRID%DplGridNodeName%][org.apache.ignite.Ignite]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=cla
> ss o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.AssertionError: Cache group is
> not initialized [grpId=-1903385190]]]
> java.lang.AssertionError: Cache group is not initialized [grpId=-1903385190]
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.destroyEvictedPartitions(GridCacheDatabaseSharedManager.java:3350)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3262)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3053)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> 2018-06-10 17:42:34.470
> [ERROR][db-checkpoint-thread-#164%DPL_GRID%DplGridNodeName%][org.apache.ignite.Ignite]
> JVM will be halted immediately due to the failure:
> [failureCtx=FailureContext [type=SYSTEM
> _WORKER_TERMINATION, err=java.lang.AssertionError: Cache group is not
> initialized [grpId=-1903385190]]]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
