Mayuresh Gharat created KAFKA-2976: -------------------------------------- Summary: Mirror maker dies if we delete a topic from destination cluster Key: KAFKA-2976 URL: https://issues.apache.org/jira/browse/KAFKA-2976 Project: Kafka Issue Type: Bug Components: clients Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat
In datapipeline, 1) Suppose the Mirror Maker is producing to a cluster with Topic T and has 128 partitions (Partition 0 to Partition 127) . The default setting on creation of a new topic on that cluster is 8 partitions. 2) After we delete the topic, the topic gets recreated with 8 partitions (Partition 0 to Partition 7). 3) The RecordAccumulator has batches for partitions from 9 to 127. Those batches get expired and the mirror makers will die to avoid data loss. We need a way to reassign those batches (batches for Partition 9 top Partition 127) in the RecordAccumulator to the newly created Topic T with 8 partitions (Partition 0 to Partition 7). -- This message was sent by Atlassian JIRA (v6.3.4#6332)