[
https://issues.apache.org/jira/browse/KAFKA-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bale updated KAFKA-13932:
-------------------------
Description:
Kafka version: 2.8.0
Replication flow: A -> B
Topic: DL2 -> A.DL2
I use mm2 to replicate cluster A data to cluster B. In some cases, I found
consumer group offset is bigger than partition offset in cluster B which will
cause data loss of the mirror topic.
[Steps]
1. Start mm2 with the attachment: connect-mirror-maker.properties.
2.Create topic DL2, push 5 messages and consume with group g1. offset
info：offset1.png
3.Stop mm2 and push 5 messages to the topic DL2.
4. After 1 hour, the data of topic DL2 is deleted.
5.Restart mm2 and sync consumer group offset, found the consume offset is
bigger than server offset
was:
Kafka version: 2.8.0
Replication flow: A -> B
Topic: DL2 -> A.DL2
I use mm2 to replicate cluster A data to cluster B. In some cases, I found
consumer group offset is bigger than partition offset in cluster B which will
cause data loss of the mirror topic.
[Steps]
1. Start mm2 with the attachment: connect-mirror-maker.properties.
2.Create topic DL2, push 5 messages and consume with group g1.
3.Stop mm2 and push 5 messages to the topic DL2.
4. After 1 hour, the data of topic DL2 is deleted.
5.Restart mm2 and sync consumer group offset, found the consume offset is
bigger than server offset
> Replication data loss in some cases
> -----------------------------------
>
> Key: KAFKA-13932
> URL: https://issues.apache.org/jira/browse/KAFKA-13932
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 2.8.0
> Reporter: Bale
> Priority: Minor
> Attachments: connect-mirror-maker.properties
>
>
> Kafka version: 2.8.0
> Replication flow: A -> B
> Topic: DL2 -> A.DL2
>
> I use mm2 to replicate cluster A data to cluster B. In some cases, I found
> consumer group offset is bigger than partition offset in cluster B which will
> cause data loss of the mirror topic.
>
> [Steps]
> 1. Start mm2 with the attachment: connect-mirror-maker.properties.
> 2.Create topic DL2, push 5 messages and consume with group g1. offset
> info：offset1.png
> 3.Stop mm2 and push 5 messages to the topic DL2.
> 4. After 1 hour, the data of topic DL2 is deleted.
> 5.Restart mm2 and sync consumer group offset, found the consume offset is
> bigger than server offset
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
