[
https://issues.apache.org/jira/browse/KAFKA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17348063#comment-17348063
]
Guozhang Wang commented on KAFKA-12743:
---------------------------------------
I think the statement should be "Streams does not have a good integration with
MM2 for DR story yet", not that "Streams does not have a good DR story" :)
There are other sync / async mechanisms for replication data between Kafka
clusters which do not have prefix on the destination cluster's topic names,
which have been used nicely with Streams for a while.
As for MM2, I think I agree that we can improve Streams' integration with this
replicator better, for example, Streams can have a specific
`mirror-maker-failover-enabled` config which 1) would expect the internal topic
names to be prefixed while 2) still use the same topic names when fetching from
committed offset topic, upon failover to the new cluster.
> [Kafka Streams] - cluster failover for stateful Kafka Streams applications
> --------------------------------------------------------------------------
>
> Key: KAFKA-12743
> URL: https://issues.apache.org/jira/browse/KAFKA-12743
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker, streams
> Affects Versions: 2.8.0
> Reporter: Sergey Zyrianov
> Priority: Major
>
> Currently, when working with Kafka backed state stores in Kafka Streams,
> these log compacted topics are given a hardcoded name :
> _app_id-storename-changelog_
> {noformat}
> public static String storeChangelogTopic(String applicationId, String
> storeName) {
> return applicationId + "-" + storeName + STATE_CHANGELOG_TOPIC_SUFFIX;
> }{noformat}
>
> MirrorMaker2(mm2) copies these topics to remote cluster under the name
> _src-cluster-alias.app_id-storename-changelog_
>
> When streams app fails over to the remote cluster it has troubles to find
> changelog topic of its state store since it was renamed - given source
> cluster prefix by mm2.
> Whats the fix should be ? instruct mm2 to keep topic name or subscribe to
> regex *._app_id-storename-changelog_ topic name for the state's changelog.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
