[
https://issues.apache.org/jira/browse/KAFKA-20913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18109586#comment-18109586
]
Shantanu commented on KAFKA-20913:
----------------------------------
I'll take this.
*Solution:* Capture {{appendTimestamp}} when {{CompletedBatch}} is constructed
(the batch max timestamp) and return that field from {{appendTimestamp()}}. Do
not re-read {{MemoryRecords}} after the pooled buffer may have been reused.
*Tests (fail on trunk, pass after the snapshot):*
- Accumulator: drain a batch at T1, {{release()}}, append a later batch that
reuses the same buffer at T2. The first batch must still report T1.
- Raft client: two voters so HWM lags; catch the listener up past the
leader-change record; append at T1 then T2 with a single reused buffer. After
commit, the listener's first data batch must still be T1.
Existing tests miss this because they use {{MemoryPool.NONE}} (never reuses
buffers) and/or call {{appendTimestamp()}} before {{release()}}.
> KafkaRaftClient#appendBatch reads a recycled batch buffer in its async commit
> callback, producing an incorrect appendTimestamp for committed batches
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-20913
> URL: https://issues.apache.org/jira/browse/KAFKA-20913
> Project: Kafka
> Issue Type: Bug
> Reporter: Anjaneya Gupta
> Assignee: Shantanu
> Priority: Minor
>
> On the leader, KafkaRaftClient#appendBatch registers an asynchronous commit
> callback and then releases the batch's memory buffer back to the pool in a
> finally block. The callback runs later (once the batch commits) and reads
> CompletedBatch#appendTimestamp() out of that buffer. By then the buffer may
> have been reused by a newer batch, so the callback reads the newer batch's
> timestamp and delivers the wrong appendTimestamp to the commit listener.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
