I wouldn't really include a non-existent group (same as we don't care about a non-existent topic), that doesn't really matter. I think having an existing group which doesn't have an offset to checkpoint is equivalent to a topic having no records to replicate from the monitoring perspective.
I think the precise way to put it is to monitor the topics and groups picked up by the filtering logic of MM2. "The list currently replicated" is not a good definition, as an empty topic would still be interesting for monitoring purposes, even if there is no message to replicate. I think the core motivation is to capture the output of the TopicFilter/GroupFilter + the extra, built-in logic of MM2 related to filtering (e.g. internal topics are never replicated, the heartbeats topics are always replicated, and so on). This logic is too complex to reproduce in an external monitoring system, as it would need to use the exact same TopicFilter/GroupFilter configs as MM2 is using, and then implement the additional built-in logic of MM2 to finally get the topics and groups picked up by the replication. I think this would be useful in any replication setups (finding the effective list of filtered topics and groups), but especially useful when using the IdentityReplicationPolicy. One gap related to the identity policy is that we cannot find the replica topics of a specific flow, even when using MirrorClient, or having access to the source and target Kafka clusters, as the "traditional" way of finding replica topics is based on topic naming and the ReplicationPolicy. Thanks, Daniel hudeqi <16120...@bjtu.edu.cn> ezt írta (időpont: 2023. ápr. 19., Sze, 10:58): > Thanks for your reply, Daniel. > Regarding the group list, do you mean that if the group of the source > cluster has not committed an offset (the group does not exist or the group > has not committed an offset to the topic being replicated), then the > current metric cannot be collected? Then this involves the question of > motivation: Do we want to monitor the topic list and group list we > configured, or the topic list and group list that are currently being > replicated? If it is the latter, shouldn't it be detected for a group that > has not committed an offset? I don't know if I understand correctly. > > best, > hudeqi > > > > -----原始邮件----- > > 发件人: "Dániel Urbán" <urb.dani...@gmail.com> > > 发送时间: 2023-04-19 15:50:01 (星期三) > > 收件人: dev@kafka.apache.org > > 抄送: > > 主题: Re: Re: [DISCUSS] KIP-918: MM2 Topic And Group Listener > > > </urb.dani...@gmail.com>
