[ https://issues.apache.org/jira/browse/KAFKA-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dhruvil Shah reassigned KAFKA-12254: ------------------------------------ Assignee: Dhruvil Shah > MirrorMaker 2.0 creates destination topic with default configs > -------------------------------------------------------------- > > Key: KAFKA-12254 > URL: https://issues.apache.org/jira/browse/KAFKA-12254 > Project: Kafka > Issue Type: Bug > Reporter: Dhruvil Shah > Assignee: Dhruvil Shah > Priority: Major > > `MirrorSourceConnector` implements the logic for replicating data, > configurations, and other metadata between the source and destination > clusters. This includes the tasks below: > # `refreshTopicPartitions` for syncing topics / partitions from source to > destination. > # `syncTopicConfigs` for syncing topic configurations from source to > destination. > A limitation is that `computeAndCreateTopicPartitions` creates topics with > default configurations on the destination cluster. A separate async task > `syncTopicConfigs` is responsible for syncing the topic configs. Before that > sync happens, topic configurations could be out of sync between the two > clusters. > In the worst case, this could lead to data loss eg. when we have a compacted > topic being mirrored between clusters which is incorrectly created with the > default configuration of `cleanup.policy = delete` on the destination before > the configurations are sync'd via `syncTopicConfigs`. > Here is an example of the divergence: > Source Topic: > ``` > Topic: foobar PartitionCount: 1 ReplicationFactor: 1 Configs: > cleanup.policy=compact,segment.bytes=1073741824 > ``` > Destination Topic: > ``` > Topic: A.foobar PartitionCount: 1 ReplicationFactor: 1 Configs: > segment.bytes=1073741824 > ``` > A safer approach is to ensure that the right configurations are set on the > destination cluster before data is replicated to it. -- This message was sent by Atlassian Jira (v8.3.4#803005)