David Dufour created KAFKA-15102: ------------------------------------ Summary: Mirror Maker 2 - KIP690 backward compatibility Key: KAFKA-15102 URL: https://issues.apache.org/jira/browse/KAFKA-15102 Project: Kafka Issue Type: Bug Components: mirrormaker Affects Versions: 3.1.0 Reporter: David Dufour
According to KIP690, "When users upgrade an existing MM2 cluster they don’t need to change any of their current configuration as this proposal maintains the default behaviour for MM2." In particular, the naming of the topic {{mm2-offset-syncs}} was supposed to use fixed separators: {{ }}{{default}} {{String offsetSyncsTopic(String targetAlias) { }}{{return}} {{" mm2-offset-syncs."}} {{+ targetCluster + }}{{{}".internal"{}}}{{{};}{}}} The implementation is actually slightly different: [https://github.com/apache/kafka/blob/400d39bb0efaa4fdda4c0597a63a1ddd17082291/connect/mirror-client/src/main/java/org/apache/kafka/connect/mirror/DefaultReplicationPolicy.java#L84] Instead, the separator is subject to customization. As a consequence, when an upgrade is performed, if the separator is customized with replication.policy.separator, the name of this internal topic changes. It then generates issues like: Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidTopicException: Topic 'mm2-offset-syncs_bkts28_internal' collides with existing topics: mm2-offset-syncs.bkts28.internal It has been observed that the replication can then be broken sometimes several days after the upgrade (reason not identified). By deleting the old topic name, it recovers. -- This message was sent by Atlassian Jira (v8.20.10#820010)