[
https://issues.apache.org/jira/browse/IGNITE-26942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Zlenko updated IGNITE-26942:
---------------------------------
Description:
Sometimes cluster restart is failing with the following error:
{code:java}
[2025-11-05T12:08:06,354][ERROR][%icbmsnt_n_3347%MessagingService-inbound-Default-0-1][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
failureCtxId=4219cbf4-eb3e-4e13-a1d2-54301b30ed9f]
org.apache.ignite.internal.failure.StackTraceCapturingException: Failed to
update data nodes history and timers on topology change
[timestamp=HybridTimestamp [physical=2025-11-05 07:07:52:606 +0000, logical=2,
composite=115495827708706818]].
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:184)
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:161)
at
org.apache.ignite.internal.distributionzones.DataNodesManager.lambda$msInvokeWithRetry$29(DataNodesManager.java:1206)
at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$sendWithRetry$50(RaftGroupServiceImpl.java:743)
at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
at
org.apache.ignite.internal.network.DefaultMessagingService.onInvokeResponse(DefaultMessagingService.java:638)
at
org.apache.ignite.internal.network.DefaultMessagingService.handleInvokeResponse(DefaultMessagingService.java:531)
at
org.apache.ignite.internal.network.DefaultMessagingService.lambda$handleMessageFromNetwork$5(DefaultMessagingService.java:463)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.util.concurrent.CompletionException:
java.lang.ClassCastException: class java.lang.Boolean cannot be cast to class
org.apache.ignite.internal.metastorage.dsl.StatementResult (java.lang.Boolean
is in module java.base of loader 'bootstrap';
org.apache.ignite.internal.metastorage.dsl.StatementResult is in unnamed module
of loader 'app')
at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
at
java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1159)
... 13 more
Caused by: java.lang.ClassCastException: class java.lang.Boolean cannot be cast
to class org.apache.ignite.internal.metastorage.dsl.StatementResult
(java.lang.Boolean is in module java.base of loader 'bootstrap';
org.apache.ignite.internal.metastorage.dsl.StatementResult is in unnamed module
of loader 'app')
at
java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)
... 13 more
[2025-11-05T12:08:06,356][ERROR][%icbmsnt_n_3347%MessagingService-inbound-Default-0-1][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
failureCtxId=ff139ac7-8519-4562-8854-1ee016317d66]
{code}
was:
Sometimes we cluster restart is failing with the following error:
{code:java}
[2025-11-05T12:08:06,354][ERROR][%icbmsnt_n_3347%MessagingService-inbound-Default-0-1][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
failureCtxId=4219cbf4-eb3e-4e13-a1d2-54301b30ed9f]
org.apache.ignite.internal.failure.StackTraceCapturingException: Failed to
update data nodes history and timers on topology change
[timestamp=HybridTimestamp [physical=2025-11-05 07:07:52:606 +0000, logical=2,
composite=115495827708706818]].
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:184)
at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:161)
at
org.apache.ignite.internal.distributionzones.DataNodesManager.lambda$msInvokeWithRetry$29(DataNodesManager.java:1206)
at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$sendWithRetry$50(RaftGroupServiceImpl.java:743)
at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
at
org.apache.ignite.internal.network.DefaultMessagingService.onInvokeResponse(DefaultMessagingService.java:638)
at
org.apache.ignite.internal.network.DefaultMessagingService.handleInvokeResponse(DefaultMessagingService.java:531)
at
org.apache.ignite.internal.network.DefaultMessagingService.lambda$handleMessageFromNetwork$5(DefaultMessagingService.java:463)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.util.concurrent.CompletionException:
java.lang.ClassCastException: class java.lang.Boolean cannot be cast to class
org.apache.ignite.internal.metastorage.dsl.StatementResult (java.lang.Boolean
is in module java.base of loader 'bootstrap';
org.apache.ignite.internal.metastorage.dsl.StatementResult is in unnamed module
of loader 'app')
at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
at
java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1159)
... 13 more
Caused by: java.lang.ClassCastException: class java.lang.Boolean cannot be cast
to class org.apache.ignite.internal.metastorage.dsl.StatementResult
(java.lang.Boolean is in module java.base of loader 'bootstrap';
org.apache.ignite.internal.metastorage.dsl.StatementResult is in unnamed module
of loader 'app')
at
java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)
... 13 more
[2025-11-05T12:08:06,356][ERROR][%icbmsnt_n_3347%MessagingService-inbound-Default-0-1][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
failureCtxId=ff139ac7-8519-4562-8854-1ee016317d66]
{code}
> ClassCastException on cluster restart
> -------------------------------------
>
> Key: IGNITE-26942
> URL: https://issues.apache.org/jira/browse/IGNITE-26942
> Project: Ignite
> Issue Type: Bug
> Reporter: Ivan Zlenko
> Assignee: Ivan Zlenko
> Priority: Major
> Labels: ignite-3
>
> Sometimes cluster restart is failing with the following error:
> {code:java}
> [2025-11-05T12:08:06,354][ERROR][%icbmsnt_n_3347%MessagingService-inbound-Default-0-1][FailureManager]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
> failureCtxId=4219cbf4-eb3e-4e13-a1d2-54301b30ed9f]
> org.apache.ignite.internal.failure.StackTraceCapturingException: Failed to
> update data nodes history and timers on topology change
> [timestamp=HybridTimestamp [physical=2025-11-05 07:07:52:606 +0000,
> logical=2, composite=115495827708706818]].
> at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:184)
> at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:161)
> at
> org.apache.ignite.internal.distributionzones.DataNodesManager.lambda$msInvokeWithRetry$29(DataNodesManager.java:1206)
> at
> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
> at
> java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
> at
> java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$sendWithRetry$50(RaftGroupServiceImpl.java:743)
> at
> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
> at
> java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
> at
> java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
> at
> org.apache.ignite.internal.network.DefaultMessagingService.onInvokeResponse(DefaultMessagingService.java:638)
> at
> org.apache.ignite.internal.network.DefaultMessagingService.handleInvokeResponse(DefaultMessagingService.java:531)
> at
> org.apache.ignite.internal.network.DefaultMessagingService.lambda$handleMessageFromNetwork$5(DefaultMessagingService.java:463)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:840)
> Caused by: java.util.concurrent.CompletionException:
> java.lang.ClassCastException: class java.lang.Boolean cannot be cast to class
> org.apache.ignite.internal.metastorage.dsl.StatementResult (java.lang.Boolean
> is in module java.base of loader 'bootstrap';
> org.apache.ignite.internal.metastorage.dsl.StatementResult is in unnamed
> module of loader 'app')
> at
> java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
> at
> java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
> at
> java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1159)
> ... 13 more
> Caused by: java.lang.ClassCastException: class java.lang.Boolean cannot be
> cast to class org.apache.ignite.internal.metastorage.dsl.StatementResult
> (java.lang.Boolean is in module java.base of loader 'bootstrap';
> org.apache.ignite.internal.metastorage.dsl.StatementResult is in unnamed
> module of loader 'app')
> at
> java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)
> ... 13 more
> [2025-11-05T12:08:06,356][ERROR][%icbmsnt_n_3347%MessagingService-inbound-Default-0-1][FailureManager]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
> failureCtxId=ff139ac7-8519-4562-8854-1ee016317d66]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
