[
https://issues.apache.org/jira/browse/KAFKA-13732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yujie Li updated KAFKA-13732:
-----------------------------
Description:
Hey,
I recently realize consumer group offset retention behavior could improve in
kafka document.
[https://kafka.apache.org/documentation/#brokerconfigs_offsets.retention.minutes]
The scenario I have is:
# A consumer group[ConsumerGroupA] with multiple consumers subscribe multiple
topics [TopicA, TopicB]
# Remove Topic B from ConsumerGroupA, but the producer still publish message
to Topic B, ConsumerGroupA still subscribe Topic A
# After `offsets.retention.minutes` reached, ConsumerGroupA's offset for topic
B is gone.
# Re-add TopicB to ConsumerGroupA, offset reset to earliest / lastest
>From my understand this configuration more topic oriented than consumer group
>describe in the doc and I wish it could be more specific.
And here is my proposing doc change:
After a consumer group loses all its consumers (i.e. becomes empty) {*}with a
topic{*}, its offsets will be kept for this retention period before getting
discarded. For standalone consumers (using manual assignment), offsets will be
expired after the time of last commit plus this retention period.
Let me know what do you think!
Thanks,
Yujie
was:
Hey,
I recently realize consumer group offset retention behavior could improve in
kafka document.
[https://kafka.apache.org/documentation/#brokerconfigs_offsets.retention.minutes]
The scenario I have is:
# A consumer group[ConsumerGroupA] with multiple consumers subscribe multiple
topics [TopicA, TopicB]
# Remove Topic B from ConsumerGroupA, but the producer still publish message
to Topic B, ConsumerGroupA still subscribe Topic A
# After `offsets.retention.minutes` reached, ConsumerGroupA's offset for topic
B is gone.
# Re-add TopicB to ConsumerGroupA, offset reset to earliest / lastest
>From my understand this configuration more topic oriented than consumer group
>describe in the doc and I wish it could be more specific.
For example:
After a consumer group loses all its consumers (i.e. becomes empty) \{*}with a
topic{*}, its offsets will be kept for this retention period before getting
discarded. For standalone consumers (using manual assignment), offsets will be
expired after the time of last commit plus this retention period.
Let me know what do you think!
Thanks,
Yujie
> Improve offsets.retention.minutes document to specify it's consumer per topic
> instead whole consumer group
> ----------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-13732
> URL: https://issues.apache.org/jira/browse/KAFKA-13732
> Project: Kafka
> Issue Type: Improvement
> Components: documentation
> Reporter: Yujie Li
> Priority: Minor
>
> Hey,
> I recently realize consumer group offset retention behavior could improve in
> kafka document.
> [https://kafka.apache.org/documentation/#brokerconfigs_offsets.retention.minutes]
> The scenario I have is:
> # A consumer group[ConsumerGroupA] with multiple consumers subscribe
> multiple topics [TopicA, TopicB]
> # Remove Topic B from ConsumerGroupA, but the producer still publish message
> to Topic B, ConsumerGroupA still subscribe Topic A
> # After `offsets.retention.minutes` reached, ConsumerGroupA's offset for
> topic B is gone.
> # Re-add TopicB to ConsumerGroupA, offset reset to earliest / lastest
>
> From my understand this configuration more topic oriented than consumer group
> describe in the doc and I wish it could be more specific.
>
> And here is my proposing doc change:
> After a consumer group loses all its consumers (i.e. becomes empty) {*}with a
> topic{*}, its offsets will be kept for this retention period before getting
> discarded. For standalone consumers (using manual assignment), offsets will
> be expired after the time of last commit plus this retention period.
>
> Let me know what do you think!
>
> Thanks,
> Yujie
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
