Kirill Tkalenko created IGNITE-26695:
----------------------------------------
Summary: Fix node fail due to RejectedExecutionException in
MetaStorageCompactionTrigger
Key: IGNITE-26695
URL: https://issues.apache.org/jira/browse/IGNITE-26695
Project: Ignite
Issue Type: Bug
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
It was found that when stopping the
*org.apache.ignite.internal.metastorage.impl.MetaStorageCompactionTrigger*,
*org.apache.ignite.internal.failure.FailureProcessor* may be called due to
j*ava.util.concurrent.RejectedExecutionException*, this needs to be fixed
correctly.
StackTrace:
{noformat}
[15:31:05]W: [:ignite-metastorage:integrationTest]
[2025-10-13T15:31:05,273][ERROR][%imsctt_n_3344%Raft-Group-Client-3][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
failureCtxId=37e0df64-5011-411c-a6b6-8a28342e7115]
[15:31:05]W: [:ignite-metastorage:integrationTest]
org.apache.ignite.internal.failure.StackTraceCapturingException: Unknown error
on new metastorage compaction revision: 238
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:184)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:161)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.metastorage.impl.MetaStorageCompactionTrigger.lambda$onCompactionRevisionUpdateBusy$7(MetaStorageCompactionTrigger.java:308)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.metastorage.server.ReadOperationForCompactionTracker.lambda$track$0(ReadOperationForCompactionTracker.java:104)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.metastorage.impl.MetaStorageManagerImpl.lambda$withTrackReadOperationFromLeaderFuture$72(MetaStorageManagerImpl.java:1348)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:669)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:652)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$scheduleRetry$51(RaftGroupServiceImpl.java:902)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.lang.Thread.run(Thread.java:833)
[15:31:05]W: [:ignite-metastorage:integrationTest] Caused by:
java.util.concurrent.CompletionException:
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@3f082613[Not
completed, task =
java.util.concurrent.Executors$RunnableAdapter@3acacd76[Wrapped task =
java.util.concurrent.CompletableFuture$UniRun@644b952f]] rejected from
java.util.concurrent.ScheduledThreadPoolExecutor@61fc508d[Terminated, pool size
= 0, active threads = 0, queued tasks = 0, completed tasks = 451]
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:791)
[15:31:05]W: [:ignite-metastorage:integrationTest] ... 17 more
[15:31:05]W: [:ignite-metastorage:integrationTest] Caused by:
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@3f082613[Not
completed, task =
java.util.concurrent.Executors$RunnableAdapter@3acacd76[Wrapped task =
java.util.concurrent.CompletableFuture$UniRun@644b952f]] rejected from
java.util.concurrent.ScheduledThreadPoolExecutor@61fc508d[Terminated, pool size
= 0, active threads = 0, queued tasks = 0, completed tasks = 451]
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:721)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniCompletion.claim(CompletableFuture.java:572)
[15:31:05]W: [:ignite-metastorage:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:784)
[15:31:05]W: [:ignite-metastorage:integrationTest] ... 17 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
