[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868544#comment-16868544
]
Adam Bellemare commented on KAFKA-7500:
---------------------------------------
[~ryannedolan] - Thanks for driving this forward, it looks extremely useful.
I'm currently evaluating this for my company's needs. I have a few questions.
1) Are the offset translation topics included in the patch you have above? I
ask because I see that https://issues.apache.org/jira/browse/KAFKA-7815 is
still open, and I am not sure if SourceTask is the provider of this info and to
the overall status of the offset translations work.
2) How do you go about switching a consumer from one cluster to another, say in
the case of a total cluster failure? I have noted that there exists the offsets
topics that indicate the source offset and equivalent destination offset for a
consumer group. Aside from the mechanisms of redirecting the consumer to the
replica cluster, what do you do in terms of ensuring that the consumer is able
to read from this destination cluster? My first instinct is to have a service
that consumes from the offsets-sync topic and sets the offsets for each
consumer group as the new sync events come in. If this is up to date with the
latest events, then it should be possible to restart the consumer from the
checkpoint in the destination cluster under the same consumer group id.
3) Do you use timestamps at all for failing over from one cluster to another?
This is a bit of an extension to #2 above, and my current understanding is that
it is not necessary to do so. I seem to recall that in previous versions of MM
it was more common to use timestamps than offsets, since the guarantee for the
offset wasn't present.
Anyways, thanks very much for your work here. I am eager to see this get into
2.4 because I believe it will make multi-cluster much more approachable for
smaller businesses.
> MirrorMaker 2.0 (KIP-382)
> -------------------------
>
> Key: KAFKA-7500
> URL: https://issues.apache.org/jira/browse/KAFKA-7500
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect, mirrormaker
> Reporter: Ryanne Dolan
> Priority: Minor
> Fix For: 2.4.0
>
>
> Implement a drop-in replacement for MirrorMaker leveraging the Connect
> framework.
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0]
> [https://github.com/apache/kafka/pull/6295]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
