[
https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718567#comment-13718567
]
Neha Narkhede commented on KAFKA-987:
-------------------------------------
Jun,
I think what you are suggesting makes sense on startup before the consumer has
consumed any messages. However, since the offset map is a cache for what's in
zookeeper, the safest way is to keep it in sync with the zookeeper data. Before
the consumer can pull any data, it has to rebalance and while rebalancing we
read the offsets from zk anyways. So I think it is correct to update the offset
cache in addPartitionTopicInfo()
> Avoid checkpointing offsets in Kafka consumer that have not changed since the
> last commit
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-987
> URL: https://issues.apache.org/jira/browse/KAFKA-987
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Swapnil Ghike
> Assignee: Swapnil Ghike
> Labels: improvement
> Fix For: 0.8
>
> Attachments: kafka-987.patch, kafka-987-v2.patch
>
>
> We need to fix the Kafka zookeeper consumer to avoid checkpointing offsets
> that have not changed since the last offset commit. This will help reduce the
> write load on zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
