[
https://issues.apache.org/jira/browse/KAFKA-13612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482006#comment-17482006
]
Jason Gustafson edited comment on KAFKA-13612 at 1/25/22, 5:59 PM:
-------------------------------------------------------------------
I think there is a higher level question here which the community probably has
never fully agreed upon. My take (and I think Colin agrees with me) is that
__consumer_offsets __ should not be accessed by clients directly. For example,
we don't make any guarantees about the compatibility of the message format used
to write group metadata and offset commits to this topic, so any tool reading
data from it might break after an upgrade. And direct writes to the topic do
not work in general because the coordinator does not materialize the data in
its cache. What is supported is indirect access through the respective request
APIs (e.g. `OffsetCommit` and `OffsetDelete`). Is there a gap in these APIs
that requires some use cases to still access __ __consumer_offsets directly?
As for the behavior change, if we were still in the context of release 2.8, I'd
probably take the old behavior without question. But now we've done 3 releases
(3.1 is out now) with this modified behavior, so I'm inclined to step back a
little to consider how we actually want this to work.
was (Author: hachikuji):
I think there is a higher level question here which the community probably has
never fully agreed upon. My take (and I think Colin agrees with me) is that
`__consumer_offsets` should not be accessed by clients directly. For example,
we don't make any guarantees about the compatibility of the message format used
to write group metadata and offset commits to this topic, so any tool reading
data from it might break after an upgrade. And direct writes to the topic do
not work in general because the coordinator does not materialize the data in
its cache. What is supported is indirect access through the respective request
APIs (e.g. `OffsetCommit` and `OffsetDelete`). Is there a gap in these APIs
that requires some use cases to still access `__consumer_offsets` directly?
As for the behavior change, if we were still in the context of release 2.8, I'd
probably take the old behavior without question. But now we've done 3 releases
(3.1 is out now) with this modified behavior, so I'm inclined to step back a
little to consider how we actually want this to work.
> internal topics won't be created in metadataRequest when
> auto.create.topics.enable=false
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-13612
> URL: https://issues.apache.org/jira/browse/KAFKA-13612
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.8.0, 3.0.0
> Reporter: Luke Chen
> Assignee: Luke Chen
> Priority: Major
>
> In KAFKA-9751, when create internal topics through FindCoordinator or
> Metadata request, we route the topic creation request to the controller
> instead of handling by itself. We change logic in
> `KafkaApis#getTopicMetadata`, and make the internal topic won't get created
> when "auto.create.topics.enable=false`.
> h4.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
