Lucas Brutschy created KAFKA-16155:
--------------------------------------
Summary: Investigate testAutoCommitIntercept
Key: KAFKA-16155
URL: https://issues.apache.org/jira/browse/KAFKA-16155
Project: Kafka
Issue Type: Bug
Components: clients, consumer
Reporter: Lucas Brutschy
Even with KAFKA-15942, the test PlaintextConsumerTest.testAutoCommitIntercept
flakes on the the initial setup (before using interceptors, so interceptors are
unrelated here, except for being used later in the test).
The problem is that we are seeking two topic partitions to offset 10 and 20,
respectively, but when we commit, we seem to have lost one of the offsets,
likely due to a race condition.
When I output `subscriptionState.allConsumed` repeatedly, I get this output:
{code:java}
allConsumed: {topic-0=OffsetAndMetadata{offset=100, leaderEpoch=0,
metadata=''}, topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null,
metadata=''}}
seeking topic-0 to FetchPosition{offset=10, offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[localhost:58298 (id: 0 rack:
null)], epoch=0}}
seeking topic-1 to FetchPosition{offset=20, offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[localhost:58301 (id: 1 rack:
null)], epoch=0}}
allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null,
metadata=''}, topic-1=OffsetAndMetadata{offset=20, leaderEpoch=null,
metadata=''}}
allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null,
metadata=''}} allConsumed: {topic-0=OffsetAndMetadata{offset=10,
leaderEpoch=null, metadata=''}, topic-1=OffsetAndMetadata{offset=0,
leaderEpoch=null, metadata=''}} autocommit start
{topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''},
topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}}
{code}
So we after we seek to 10 / 20, we lose one of the offsets, maybe because we
haven't reconciled the assignment yet. Later, we get the second topic partition
assigned, but the offset is initialized to 0.
We should investigate whether this can be made more like the behavior in the
original consumer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
