[
https://issues.apache.org/jira/browse/KAFKA-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758830#comment-13758830
]
Tejas Patil commented on KAFKA-1012:
------------------------------------
Sriram and Neha had concerns about the OffsetManager interface in their review
comments.
If we don't want other offset manager implementations to store the offsets in
logs, then it will be only the inbuilt offset manager that would need the
method for loading of offsets. Then, OffsetManager interface need NOT have the
method for loading offsets as it is too specific to the inbuilt offset manager
and pollutes the interface. The inbuilt offset manager needs a notification
whenever there is change in the leadership for the offset topic partition so
that it starts loading offsets. One solution in my mind: Have the inbuilt
offset manager create a watcher to the Zk path where the offset topics' info is
stored. On updates, default offset manager checks if its current broker has
been newly elected as the leader (a cache of its earlier ownership must be
stored) and then start loading offsets. This would end up adding a dependency
on Zk.
> Implement an Offset Manager and hook offset requests to it
> ----------------------------------------------------------
>
> Key: KAFKA-1012
> URL: https://issues.apache.org/jira/browse/KAFKA-1012
> Project: Kafka
> Issue Type: Sub-task
> Components: consumer
> Reporter: Tejas Patil
> Assignee: Tejas Patil
> Priority: Minor
> Attachments: KAFKA-1012.patch, KAFKA-1012-v2.patch
>
>
> After KAFKA-657, we have a protocol for consumers to commit and fetch offsets
> from brokers. Currently, consumers are not using this API and directly
> talking with Zookeeper.
> This Jira will involve following:
> 1. Add a special topic in kafka for storing offsets
> 2. Add an OffsetManager interface which would handle storing, accessing,
> loading and maintaining consumer offsets
> 3. Implement offset managers for both of these 2 choices : existing ZK based
> storage or inbuilt storage for offsets.
> 4. Leader brokers would now maintain an additional hash table of offsets for
> the group-topic-partitions that they lead
> 5. Consumers should now use the OffsetCommit and OffsetFetch API
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
