shashank created CAMEL-27316:
--------------------------------

             Summary: camel-kafka - with batching, the records held for an 
incomplete batch are committed after their partition was revoked, which moves 
back the new owner's offset
                 Key: CAMEL-27316
                 URL: https://issues.apache.org/jira/browse/CAMEL-27316
             Project: Camel
          Issue Type: Bug
          Components: camel-kafka
            Reporter: shashank
            Assignee: shashank


h3. What happens

With {{batching=true}} the consumer holds the records of several polls until 
the batch is full ({{maxPollRecords}}) or {{pollTimeoutMs}} has passed 
({{KafkaRecordBatchingProcessor}}). When a rebalance revokes a partition while 
some of its records are held, {{ClassicRebalanceListener.onPartitionsRevoked}} 
only commits the {{OffsetCache}}; the held records stay in the batch. The batch 
is dispatched later, and its {{CommitSynchronization}} commits the revoked 
partition ({{recordOffset}} + {{commit(partition)}}):

# consumer A polls records 0 to 2 of partition P and holds them;
# a rebalance gives P to consumer B, which reads from the committed offset (0), 
processes up to 9 and commits 10;
# A's batch is dispatched: A processes 0 to 2 again and commits 3. The 
committed offset goes back from 10 to 3, and records 3 to 9 are read again by 
the next owner.

This happens with the default options: with batching, 
{{KafkaConfiguration.isAutoCommitEnable()}} is false (the client does not auto 
commit) and Camel's {{AsyncCommitManager}} commits each batch. The classic 
group protocol checks the generation of a commit, not which partitions the 
member owns, so the commit is accepted. With an eager rebalance the same 
consumer can also get P back, read 0 to 2 again, and hold them twice in one 
batch.

CAMEL-26978 handled the same case for asynchronous manual commits (a commit 
requested after the partition was revoked is not committed).

h3. How to reproduce

{{KafkaBatchingConsumerRevokeTest}} (camel-kafka, {{MockConsumer}}): hold 3 
records of partition 0, revoke it, let the new owner commit 10, then fill the 
batch with records of partition 1: the committed offset of partition 0 goes 
back to 3 and the batch holds the records of the revoked partition.

Found with a TLA+ model of the batching consumer.

h3. Fix

On a revoke (or loss) of partitions, the rebalance listener wrapper of 
{{KafkaFetchRecords}} asks the record processor to drop the held records of 
those partitions ({{KafkaRecordProcessorFacade.discardPendingRecords}}, a new 
default method; the batching facade removes and releases the exchanges). The 
next owner reads them from the committed offset.

_Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to