[
https://issues.apache.org/jira/browse/KAFKA-17851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17892865#comment-17892865
]
Greg Harris commented on KAFKA-17851:
-------------------------------------
[~firebook] I'm still not understanding the use case.
If you have a group consumerB reading from A.topic1, that data is not present
elsewhere, including in the MM2 internal topics. MM2 only supports the "forward
sync" i.e. from consumerA on cluster A to consumerA on cluster B.
If a group consumerA on the A cluster is being synced to the B cluster, and the
B cluster is offline and rebuilt/replaced by cluster C, MM2 will resync the
group as mirroring proceeds, as all necessary data is present on cluster A.
Your use-case sounds more like you need a replication solution for consumerB,
which is being discussed in a very recent KIP:
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-1098%3A+Reverse+Checkpointing+in+MirrorMaker]
Perhaps there is a suitable solution where consumerB is replicated back to
cluster A, and then consumerB is synced from cluster A to cluster C with a
normal forward-sync. I have not thought it completely through, or reviewed the
KIP in detail to see if that solution is viable.
Please get involved in the KIP discussion if you're interested.
> Save internal topic to a separate kafka cluster
> -----------------------------------------------
>
> Key: KAFKA-17851
> URL: https://issues.apache.org/jira/browse/KAFKA-17851
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker
> Reporter: Zhijian Chen
> Priority: Major
>
> For the mm2-offset-syncs topic there is a parameter to control its storage
> cluster: offset-syncs.topic.location, but this parameter can only control
> whether it is in the source or target cluster.
> For checkpoints topic and heartbeat topic, there is no parameter to control
> their storage clusters.
> In a multi-cluster backup scenario, we would like to store these internal
> topics in a separate kafka cluster, which has the advantage of being easy to
> manage, and we can deploy this separate cluster across three server rooms so
> that we don't have to worry about the failure of this separate cluster, which
> also solves the problem of the existing scenario, where the internal topics
> cannot be accessed due to the failure of the source or target clusters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
