[
https://issues.apache.org/jira/browse/IGNITE-25191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denis Chudov updated IGNITE-25191:
----------------------------------
Description:
Messages like the following ones are repeated hundreds of times:
{code:java}
[16:22:20]W: [:ignite-sql-engine:integrationTest]
[2025-04-17T12:22:20,425][ERROR][%sqllogic0%partition-operations-13][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR]
[16:22:20]W: [:ignite-sql-engine:integrationTest]
org.apache.ignite.internal.failure.FailureManager$StackTraceCapturingException:
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
IGN-REP-3 TraceId:bf57f91b-7981-431a-bad7-ac8e829a4569 Could not wait for the
replica readiness due to timeout [replicaGroupId=TablePartitionIdMessageImpl
[partitionId=13, tableId=62], req=VacuumTxStateReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:163)
~[ignite-failure-handler-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:140)
~[ignite-failure-handler-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.tx.impl.PersistentTxStateVacuumizer.lambda$vacuumPersistentTxStates$0(PersistentTxStateVacuumizer.java:154)
~[ignite-transactions-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.replicator.ReplicaService.lambda$sendToReplicaRaw$5(ReplicaService.java:204)
~[ignite-replicator-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.lang.Thread.run(Thread.java:833) [?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
java.util.concurrent.CompletionException:
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
IGN-REP-3 TraceId:bf57f91b-7981-431a-bad7-ac8e829a4569 Could not wait for the
replica readiness due to timeout [replicaGroupId=TablePartitionIdMessageImpl
[partitionId=13, tableId=62], req=VacuumTxStateReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] ... 9 more
[16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
Could not wait for the replica readiness due to timeout
[replicaGroupId=TablePartitionIdMessageImpl [partitionId=13, tableId=62],
req=VacuumTxStateReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.ExceptionUtils.lambda$withCause$1(ExceptionUtils.java:511)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.ExceptionUtils.withCauseInternal(ExceptionUtils.java:576)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.ExceptionUtils.withCause(ExceptionUtils.java:511)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] ... 7 more
[16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
java.util.concurrent.TimeoutException: Invocation timed out
[message=org.apache.ignite.internal.replicator.message.AwaitReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.future.timeout.TimeoutWorker.body(TimeoutWorker.java:90)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.worker.IgniteWorker.run(IgniteWorker.java:97)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] ... 1 more{code}
It seems that some vacuum-related activity consistently fails.
was:
Messages like the following ones are repeated hundreds of times:
[16:22:20]W: [:ignite-sql-engine:integrationTest]
[2025-04-17T12:22:20,425][ERROR][%sqllogic0%partition-operations-13][FailureManager]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR]
[16:22:20]W: [:ignite-sql-engine:integrationTest]
org.apache.ignite.internal.failure.FailureManager$StackTraceCapturingException:
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
IGN-REP-3 TraceId:bf57f91b-7981-431a-bad7-ac8e829a4569 Could not wait for the
replica readiness due to timeout [replicaGroupId=TablePartitionIdMessageImpl
[partitionId=13, tableId=62], req=VacuumTxStateReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:163)
~[ignite-failure-handler-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:140)
~[ignite-failure-handler-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.tx.impl.PersistentTxStateVacuumizer.lambda$vacuumPersistentTxStates$0(PersistentTxStateVacuumizer.java:154)
~[ignite-transactions-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.replicator.ReplicaService.lambda$sendToReplicaRaw$5(ReplicaService.java:204)
~[ignite-replicator-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.lang.Thread.run(Thread.java:833) [?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
java.util.concurrent.CompletionException:
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
IGN-REP-3 TraceId:bf57f91b-7981-431a-bad7-ac8e829a4569 Could not wait for the
replica readiness due to timeout [replicaGroupId=TablePartitionIdMessageImpl
[partitionId=13, tableId=62], req=VacuumTxStateReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636)
~[?:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] ... 9 more
[16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
Could not wait for the replica readiness due to timeout
[replicaGroupId=TablePartitionIdMessageImpl [partitionId=13, tableId=62],
req=VacuumTxStateReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.ExceptionUtils.lambda$withCause$1(ExceptionUtils.java:511)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.ExceptionUtils.withCauseInternal(ExceptionUtils.java:576)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.ExceptionUtils.withCause(ExceptionUtils.java:511)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] ... 7 more
[16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
java.util.concurrent.TimeoutException: Invocation timed out
[message=org.apache.ignite.internal.replicator.message.AwaitReplicaRequestImpl]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.future.timeout.TimeoutWorker.body(TimeoutWorker.java:90)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] at
org.apache.ignite.internal.util.worker.IgniteWorker.run(IgniteWorker.java:97)
~[ignite-core-3.1.0-SNAPSHOT.jar:?]
[16:22:20]W: [:ignite-sql-engine:integrationTest] ... 1 more
It seems that some vacuum-related activity consistently fails.
> Vacuum silently fails in ItSqlLogicTest
> ---------------------------------------
>
> Key: IGNITE-25191
> URL: https://issues.apache.org/jira/browse/IGNITE-25191
> Project: Ignite
> Issue Type: Bug
> Reporter: Roman Puchkovskiy
> Priority: Major
> Labels: ignite-3
> Attachments:
> _Integration_Tests_Module_SQL_Engine_SQL_Logic_1_17837.log.zip
>
>
> Messages like the following ones are repeated hundreds of times:
>
> {code:java}
> [16:22:20]W: [:ignite-sql-engine:integrationTest]
> [2025-04-17T12:22:20,425][ERROR][%sqllogic0%partition-operations-13][FailureManager]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR]
> [16:22:20]W: [:ignite-sql-engine:integrationTest]
> org.apache.ignite.internal.failure.FailureManager$StackTraceCapturingException:
> org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
> IGN-REP-3 TraceId:bf57f91b-7981-431a-bad7-ac8e829a4569 Could not wait for the
> replica readiness due to timeout [replicaGroupId=TablePartitionIdMessageImpl
> [partitionId=13, tableId=62], req=VacuumTxStateReplicaRequestImpl]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:163)
> ~[ignite-failure-handler-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:140)
> ~[ignite-failure-handler-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.tx.impl.PersistentTxStateVacuumizer.lambda$vacuumPersistentTxStates$0(PersistentTxStateVacuumizer.java:154)
> ~[ignite-transactions-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
> ~[?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
> ~[?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
> ~[?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
> ~[?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.replicator.ReplicaService.lambda$sendToReplicaRaw$5(ReplicaService.java:204)
> ~[ignite-replicator-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
> [?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
> [?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
> [?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> [?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> [?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.lang.Thread.run(Thread.java:833) [?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
> java.util.concurrent.CompletionException:
> org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
> IGN-REP-3 TraceId:bf57f91b-7981-431a-bad7-ac8e829a4569 Could not wait for the
> replica readiness due to timeout [replicaGroupId=TablePartitionIdMessageImpl
> [partitionId=13, tableId=62], req=VacuumTxStateReplicaRequestImpl]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
> ~[?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
> ~[?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636)
> ~[?:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] ... 9 more
> [16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
> org.apache.ignite.internal.replicator.exception.ReplicationTimeoutException:
> Could not wait for the replica readiness due to timeout
> [replicaGroupId=TablePartitionIdMessageImpl [partitionId=13, tableId=62],
> req=VacuumTxStateReplicaRequestImpl]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.util.ExceptionUtils.lambda$withCause$1(ExceptionUtils.java:511)
> ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.util.ExceptionUtils.withCauseInternal(ExceptionUtils.java:576)
> ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.util.ExceptionUtils.withCause(ExceptionUtils.java:511)
> ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] ... 7 more
> [16:22:20]W: [:ignite-sql-engine:integrationTest] Caused by:
> java.util.concurrent.TimeoutException: Invocation timed out
> [message=org.apache.ignite.internal.replicator.message.AwaitReplicaRequestImpl]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.future.timeout.TimeoutWorker.body(TimeoutWorker.java:90)
> ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] at
> org.apache.ignite.internal.util.worker.IgniteWorker.run(IgniteWorker.java:97)
> ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> [16:22:20]W: [:ignite-sql-engine:integrationTest] ... 1 more{code}
>
> It seems that some vacuum-related activity consistently fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
