[
https://issues.apache.org/jira/browse/KAFKA-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18106795#comment-18106795
]
Szymon Blumczynski commented on KAFKA-18535:
--------------------------------------------
This is a duplicate of KAFKA-20112, which fixed the same test and is resolved.
Root cause: the test used awaitRebalance() as a proxy for "assignment
complete", but an assignment does not imply an initialized fetch position.
Since the test produced no records, topic-0 could still be in INITIALIZING when
the revocation began. Both protocols mark partitions pendingRevocation before
invoking onPartitionsRevoked
(AbstractMembershipManager.markPendingRevocationToPauseFetching,
ConsumerCoordinator.onJoinPrepare), and
SubscriptionState.TopicPartitionState.shouldInitialize excludes
pendingRevocation partitions, so the consumer.position(tp) call inside the
callback could never resolve a position. It spun for the full
default.api.timeout.ms and threw the reported "Timeout of 60000ms expired
before the position for partition topic-0 could be determined".
That exception was then swallowed by
ConsumerRebalanceListenerInvoker.invokePartitionsRevoked (returns rather than
throws) and the rebalance continued, so onPartitionsAssigned still fired,
awaitRebalance returned, and the assertion saw the untouched -1 sentinel. The
still-unpositioned partition resurfaced as the suppressed timeout during
close() in tearDown.
Fixed by commit [3d8a90a47b
|https://github.com/apache/kafka/commit/3d8a90a47b5ed3c4305d84e5495eca66ed3060d0](PR
[#21746|https://github.com/apache/kafka/pull/21746]), which produces a record
per topic and replaces both awaitRebalance() calls with awaitNonEmptyRecords(),
guaranteeing a valid position before revocation.
Verified on trunk
[@05f61b9ff5|https://github.com/apache/kafka/commit/05f61b9ff5ed72f32ef8371f6ae338ab9b8f7ce5]:
30 repeats each of testClassicConsumerMaxPollIntervalMsDelayInRevocation and
testAsyncConsumerMaxPollIntervalMsDelayInRevocation via
-Pkafka.cluster.test.repeat=30 - 0 occurrences of this failure mode.
Hey [~yangpoan] I believe the ticket can be closed.
> Flaky PlaintextConsumerPollTest.testMaxPollIntervalMsDelayInRevocation
> ----------------------------------------------------------------------
>
> Key: KAFKA-18535
> URL: https://issues.apache.org/jira/browse/KAFKA-18535
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: PoAn Yang
> Priority: Major
> Labels: integration-test, kip-848-client-support
>
> This case failed 6 times with a same error since from Jan 10 2025.
>
> {noformat}
> org.opentest4j.AssertionFailedError: expected: <0> but was: <-1>
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at
> app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
> at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:166)
> at
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:161)
> at
> app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:632)
> at
> app//kafka.api.PlaintextConsumerPollTest.testMaxPollIntervalMsDelayInRevocation(PlaintextConsumerPollTest.scala:128)
> at [email protected]/java.lang.reflect.Method.invoke(Method.java:580)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:197)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
> at
> [email protected]/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:807)
> at
> [email protected]/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:294)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
> at
> [email protected]/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
> at
> [email protected]/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> [email protected]/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
> at
> [email protected]/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
> at
> [email protected]/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:294)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
> at
> [email protected]/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
> at
> [email protected]/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
> at
> [email protected]/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> [email protected]/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
> at
> [email protected]/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
> at
> [email protected]/java.util.stream.ReferencePipeline$7$1FlatMap.accept(ReferencePipeline.java:294)
> at
> [email protected]/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
> at
> [email protected]/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:570)
> at
> [email protected]/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> [email protected]/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
> at
> [email protected]/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
> at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1597)
> at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1597)
> Suppressed: org.apache.kafka.common.KafkaException: Failed to close
> kafka consumer
> at
> app//org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.close(AsyncKafkaConsumer.java:1364)
> at
> app//org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.close(AsyncKafkaConsumer.java:1251)
> at
> app//org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1795)
> at
> app//kafka.api.IntegrationTestHarness.$anonfun$tearDown$3(IntegrationTestHarness.scala:243)
> at
> app//kafka.api.IntegrationTestHarness.$anonfun$tearDown$3$adapted(IntegrationTestHarness.scala:243)
> at
> app//scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
> at
> app//scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
> at
> app//scala.collection.AbstractIterable.foreach(Iterable.scala:935)
> at
> app//kafka.api.IntegrationTestHarness.tearDown(IntegrationTestHarness.scala:243)
> ... 45 more
> Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout of
> 60000ms expired before the position for partition topic-0 could be
> determined{noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
