[ https://issues.apache.org/jira/browse/KAFKA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772847#comment-17772847 ]
Greg Harris commented on KAFKA-15564: ------------------------------------- Hi [~hemanthsavasere] the offsets translation behavior was changed in KAFKA-12468 to avoid a data loss bug. You will need to lower the `offset.lag.max` if you want more precise consumer offset translation at the end of the topic. > Kafka 3.5.1- Mirror Maker 2 replicating the wrong consumer group offsets in > destination > ---------------------------------------------------------------------------------------- > > Key: KAFKA-15564 > URL: https://issues.apache.org/jira/browse/KAFKA-15564 > Project: Kafka > Issue Type: Bug > Components: mirrormaker > Affects Versions: 3.5.1 > Reporter: Hemanth Savasere > Priority: Major > Attachments: Destination Consumer Group Offsets Describe.txt, MM2 > 3.5.1 Logs.txt, Source Consumer Group Offsets Describe.txt, > docker-compose.yml, mm2.properties > > > Issue : Mirror Maker 2 not replicating the proper consumer group offsets when > replication happens from source to destination kafka brokers > Steps to Reproduce : > # Created 2 Kafka clusters locally using the attached docker-compose.yml > file. Was using the confluent platform 7.2.1 docker images > # Had added the below section in docker file to create 20 topics and produce > randomly between 1000 to 2000 messages in each topic, and then consume the > same messages using consumer-groups. > {code:java} > /etc/confluent/docker/run & > sleep 20 > for i in {1..20}; do > kafka-topics --create --bootstrap-server kafka-1:9092 > --replication-factor 1 --partitions 1 --topic topic$$i > done > for i in {1..20}; do > num_msgs=$$(shuf -i 1000-2000 -n 1) > seq 1 $$num_msgs | kafka-console-producer --broker-list > kafka-1:9092 --topic topic$$i > done > for i in {1..20}; do > timeout 10 kafka-console-consumer --bootstrap-server kafka-1:9092 > --topic topic$$i --group consumer-group$$i > done > wait > {code} > # Ran the Mirror Maker 2 using the connect-mirror-maker.sh script after > downloading the Kafka 3.5.1 release binaries. Verified the 3.5.1 version was > running and commitID 2c6fb6c54472e90a was shown in attached MM2 logs file. > NOTE : Have attached the Kafka Docker file, the mirror maker 2 logs and > mirror maker 2 properties file. Also, have attached the describe command > output of all the consumer groups present in source and destination. -- This message was sent by Atlassian Jira (v8.20.10#820010)