kaushik srinivas created KAFKA-17101: ----------------------------------------
Summary: Mirror maker internal topics cleanup policy changes to 'delete' from 'compact' Key: KAFKA-17101 URL: https://issues.apache.org/jira/browse/KAFKA-17101 Project: Kafka Issue Type: Bug Affects Versions: 3.6.1, 3.5.1, 3.4.1 Reporter: kaushik srinivas Scenario/Setup details Kafka cluster 1: 3 replicas Kafka cluster 2: 3 replicas MM1 moving data from cluster 1 to cluster 2 MM2 moving data from cluster 2 to cluster 1 Sometimes with a reboot of the kafka cluster 1 and MM1 instance, we observe MM failing to come up with below exception, {code:java} {"message":"DistributedHerder-connect-1-1 - org.apache.kafka.connect.runtime.distributed.DistributedHerder - [Worker clientId=connect-1, groupId=site1-mm2] Uncaught exception in herder work thread, exiting: "}} org.apache.kafka.common.config.ConfigException: Topic 'mm2-offsets.site1.internal' supplied via the 'offset.storage.topic' property is required to have 'cleanup.policy=compact' to guarantee consistency and durability of source connector offsets, but found the topic currently has 'cleanup.policy=delete'. Continuing would likely result in eventually losing source connector offsets and problems restarting this Connect cluster in the future. Change the 'offset.storage.topic' property in the Connect worker configurations to use a topic with 'cleanup.policy=compact'. {code} Once the topic is altered with cleanup policy of compact. MM works just fine. This is happening on our setups sporadically and across varieties of scenarios. Not been successful in identifying the exact reproduction steps as of now. -- This message was sent by Atlassian Jira (v8.20.10#820010)