[
https://issues.apache.org/jira/browse/KAFKA-12254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dhruvil Shah reassigned KAFKA-12254:
------------------------------------
Assignee: Dhruvil Shah
> MirrorMaker 2.0 creates destination topic with default configs
> --------------------------------------------------------------
>
> Key: KAFKA-12254
> URL: https://issues.apache.org/jira/browse/KAFKA-12254
> Project: Kafka
> Issue Type: Bug
> Reporter: Dhruvil Shah
> Assignee: Dhruvil Shah
> Priority: Major
>
> `MirrorSourceConnector` implements the logic for replicating data,
> configurations, and other metadata between the source and destination
> clusters. This includes the tasks below:
> # `refreshTopicPartitions` for syncing topics / partitions from source to
> destination.
> # `syncTopicConfigs` for syncing topic configurations from source to
> destination.
> A limitation is that `computeAndCreateTopicPartitions` creates topics with
> default configurations on the destination cluster. A separate async task
> `syncTopicConfigs` is responsible for syncing the topic configs. Before that
> sync happens, topic configurations could be out of sync between the two
> clusters.
> In the worst case, this could lead to data loss eg. when we have a compacted
> topic being mirrored between clusters which is incorrectly created with the
> default configuration of `cleanup.policy = delete` on the destination before
> the configurations are sync'd via `syncTopicConfigs`.
> Here is an example of the divergence:
> Source Topic:
> ```
> Topic: foobar PartitionCount: 1 ReplicationFactor: 1 Configs:
> cleanup.policy=compact,segment.bytes=1073741824
> ```
> Destination Topic:
> ```
> Topic: A.foobar PartitionCount: 1 ReplicationFactor: 1 Configs:
> segment.bytes=1073741824
> ```
> A safer approach is to ensure that the right configurations are set on the
> destination cluster before data is replicated to it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
