[
https://issues.apache.org/jira/browse/KAFKA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismael Juma updated KAFKA-5443:
-------------------------------
Status: Patch Available (was: Open)
> Consumer should use last offset from batch to set next fetch offset
> -------------------------------------------------------------------
>
> Key: KAFKA-5443
> URL: https://issues.apache.org/jira/browse/KAFKA-5443
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, core, producer
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Fix For: 0.11.0.0
>
>
> With message format v2, the log cleaner preserves the last offset in each
> batch even if the last record is removed. Currently when the batch is
> consumed by the consumer, we use the last record in the batch to determine
> the next offset to fetch. So if the last record in the batch was removed
> through compaction, the next fetch offset will still point to an offset in
> the current batch and it will be refetched. In the worst case, if the fetch
> size has room for that batch, the consumer will not be able to make progress.
> To fix this, we should advance the next fetch offset to the last offset from
> the batch once we have consumed that batch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
