[
https://issues.apache.org/jira/browse/KAFKA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338517#comment-17338517
]
Sergey Zyrianov commented on KAFKA-12743:
-----------------------------------------
Hi [~guozhang],
We do manual failover to another cluster by restarting the app with a new set
of brokers. App does stream("*.app-topic") and has changelog with the state.
We can change the app-name but then we will loose offset positions which are
kept at sync b/w clusters by mm2.
Also reverting back to original cluster - we need to change then app-name to -
I dont have a good option.
We are thinking about a workaround to replace statestore and its internal topic
with a ktable join, where ktable is built from
stream(.*.topic-we-push-state-to-manually). join to replace store.get and
store.put is replaced with pushing state directly to
"topic-we-push-state-to-manually". Then we exclude ktable topic from mm2
replication list :
# Streams(.*.state-topic).totable(“state-table-dont-mm2-me”)
# Join(.*.app-topic, state-table-dont-mm2-me).process(k, v-> push new state to
state-topic)
> [Kafka Streams] - regex name for state-store change-log topic
> -------------------------------------------------------------
>
> Key: KAFKA-12743
> URL: https://issues.apache.org/jira/browse/KAFKA-12743
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker, streams
> Affects Versions: 2.8.0
> Reporter: Sergey Zyrianov
> Priority: Major
>
> Currently, when working with Kafka backed state stores in Kafka Streams,
> these log compacted topics are given a hardcoded name :
> _app_id-storename-changelog_
> {noformat}
> public static String storeChangelogTopic(String applicationId, String
> storeName) {
> return applicationId + "-" + storeName + STATE_CHANGELOG_TOPIC_SUFFIX;
> }{noformat}
>
> MirrorMaker2(mm2) copies these topics to remote cluster under the name
> _src-cluster-alias.app_id-storename-changelog_
>
> When streams app fails over to the remote cluster it has troubles to find
> changelog topic of its state store since it was renamed - given source
> cluster prefix by mm2.
> Whats the fix should be ? instruct mm2 to keep topic name or subscribe to
> regex *._app_id-storename-changelog_ topic name for the state's changelog.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
