[
https://issues.apache.org/jira/browse/IGNITE-27146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Efremov updated IGNITE-27146:
-------------------------------------
Summary: Critical system error detected with Failed to update distribution
zones' logical topology and version keys (was: Critical system error detected
with Failed to update distribution zones' logical topology and version keys on
ItTxResourcesVacuumTest#testRecoveryAfterPersistentStateVacuumized)
> Critical system error detected with Failed to update distribution zones'
> logical topology and version keys
> -----------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-27146
> URL: https://issues.apache.org/jira/browse/IGNITE-27146
> Project: Ignite
> Issue Type: Bug
> Reporter: Mikhail Efremov
> Assignee: Mikhail Efremov
> Priority: Major
> Labels: MakeTeamcityGreenAgain, ignite-3
> Attachments: _Integration_Tests_Integration_Transactions_366.log.zip
>
>
> *Subject test*
> {{ItTxResourcesVacuumTest#testRecoveryAfterPersistentStateVacuumized}}
> {{ItClusterIdChangeTest#nodesWithDifferentClusterIdsCannotCommunicate}}
> *Reason to fail*
> {code:java}
> [2025-11-19T00:45:54,257][ERROR][%itrvt_trapsv_3346%Raft-Group-Client-7][FailureManager]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR,
> failureCtxId=53f58e1c-43ce-42a5-8f5e-c7ae557fc362]
>
> org.apache.ignite.internal.failure.StackTraceCapturingException: Failed to
> update distribution zones' logical topology and version keys [topology =
> [{id=fa8125d0-c1aa-467c-9f6d-a389d02dff35, name=itrvt_trapsv_3346,
> address=127.0.1.1:3346}, {id=71992d35-e120-4efd-bb8e-d12899d195ea,
> name=itrvt_trapsv_3344, address=127.0.1.1:3344}], version = 4]
> at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:192)
> at
> org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:169)
> at
> org.apache.ignite.internal.distributionzones.DistributionZoneManager.lambda$updateLogicalTopologyInMetaStorage$8(DistributionZoneManager.java:509)
> at
> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
> at
> java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at
> java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2090)
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:718)
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.sendWithRetry(RaftGroupServiceImpl.java:692)
> at
> org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$scheduleRetry$51(RaftGroupServiceImpl.java:944)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.util.concurrent.CompletionException:
> java.util.concurrent.TimeoutException: Send with retry timed out [retryCount
> = 150, groupId = metastorage_group, traceId = null, request =
> org.apache.ignite.raft.jraft.rpc.WriteActionRequestImpl(org.apache.ignite.internal.metastorage.command.MultiInvokeCommandImpl),
> originCommand = null, retryReasons = [[time=1763534749244, msg=Peer
> itrvt_trapsv_3346:0 returned code EPERM: Is not leader.;
> attemptWaitDuration=193, attemptDuration=7,
> attemptStartTime=2025-11-19T00:45:49,244], [time=1763534749445, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=193,
> attemptDuration=8, attemptStartTime=2025-11-19T00:45:49,445],
> [time=1763534749645, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=192, attemptDuration=8,
> attemptStartTime=2025-11-19T00:45:49,645], [time=1763534749846, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=192,
> attemptDuration=9, attemptStartTime=2025-11-19T00:45:49,846],
> [time=1763534750046, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=191, attemptDuration=9,
> attemptStartTime=2025-11-19T00:45:50,046], [time=1763534750246, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=191,
> attemptDuration=9, attemptStartTime=2025-11-19T00:45:50,246],
> [time=1763534750447, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=201, attemptDuration=0,
> attemptStartTime=2025-11-19T00:45:50,447], [time=1763534750647, msg=Peer
> itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=200,
> attemptDuration=0, attemptStartTime=2025-11-19T00:45:50,647],
> [time=1763534750848, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=200, attemptDuration=1,
> attemptStartTime=2025-11-19T00:45:50,848], [time=1763534751048, msg=Peer
> itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=199,
> attemptDuration=1, attemptStartTime=2025-11-19T00:45:51,048],
> [time=1763534751249, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=199, attemptDuration=2,
> attemptStartTime=2025-11-19T00:45:51,249], [time=1763534751449, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=198,
> attemptDuration=2, attemptStartTime=2025-11-19T00:45:51,449],
> [time=1763534751650, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=198, attemptDuration=3,
> attemptStartTime=2025-11-19T00:45:51,650], [time=1763534751850, msg=Peer
> itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=197,
> attemptDuration=3, attemptStartTime=2025-11-19T00:45:51,850],
> [time=1763534752050, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=197, attemptDuration=3,
> attemptStartTime=2025-11-19T00:45:52,050], [time=1763534752251, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=197,
> attemptDuration=4, attemptStartTime=2025-11-19T00:45:52,251],
> [time=1763534752451, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=196, attemptDuration=4,
> attemptStartTime=2025-11-19T00:45:52,451], [time=1763534752651, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=196,
> attemptDuration=4, attemptStartTime=2025-11-19T00:45:52,651],
> [time=1763534752852, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=196, attemptDuration=5,
> attemptStartTime=2025-11-19T00:45:52,852], [time=1763534753052, msg=Peer
> itrvt_trapsv_3345:0 threw PeerUnavailableException; attemptWaitDuration=195,
> attemptDuration=5, attemptStartTime=2025-11-19T00:45:53,052],
> [time=1763534753252, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=195, attemptDuration=5,
> attemptStartTime=2025-11-19T00:45:53,252], [time=1763534753452, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=195,
> attemptDuration=5, attemptStartTime=2025-11-19T00:45:53,452],
> [time=1763534753653, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=195, attemptDuration=6,
> attemptStartTime=2025-11-19T00:45:53,653], [time=1763534753853, msg=Peer
> itrvt_trapsv_3344:0 threw PeerUnavailableException; attemptWaitDuration=194,
> attemptDuration=6, attemptStartTime=2025-11-19T00:45:53,853],
> [time=1763534754054, msg=Peer itrvt_trapsv_3346:0 returned code EPERM: Is not
> leader.; attemptWaitDuration=194, attemptDuration=7,
> attemptStartTime=2025-11-19T00:45:54,054]], stopTime = 1763534754193,
> currentTime = 1763534754254, startTime = 1763534724194, duration = 30060].
> {code}
> *Failed TC build*
> [link|https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3x_Test_RunAllTests/9661272]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
