[
https://issues.apache.org/jira/browse/KAFKA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489631#comment-17489631
]
Mickael Maison commented on KAFKA-12635:
----------------------------------------
Looking at this again (sorry for the delay).
The offset on the target being negative should not have a functional impact on
the consumer. The offset is "out of range" so the auto.offset.reset
configuration will be used to find a new valid position. As there are no
records in the target partition, whether the consumer resets to latest or
earliest will have make no difference and it will set its position to 0.
But I understand it may be annoying in terms of metrics. I guess in theory it
could also lead to records being skipped if suddenly records are produced to
the source cluster and we start a consumer with auto.offset.reset to latest on
the target cluster before MirrorMaker is able to emit a new checkpoint/commit
offsets.
I think a better alternative than resetting the offset to 0 is to actually not
commit any offsets in the target cluster until some records have been mirrored.
> Mirrormaker 2 offset sync is incorrect if the target partition is empty
> -----------------------------------------------------------------------
>
> Key: KAFKA-12635
> URL: https://issues.apache.org/jira/browse/KAFKA-12635
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 2.7.0
> Reporter: Frank Yi
> Assignee: Ning Zhang
> Priority: Major
>
> This bug occurs when using Mirrormaker with "sync.group.offsets.enabled =
> true".
> If a source partition is empty, but the source consumer group's offset for
> that partition is non-zero, then Mirrormaker sets the target consumer group's
> offset for that partition to the literal, not translated, offset of the
> source consumer group. This state can be reached if the source consumer group
> consumed some records that were now deleted (like by a retention policy), or
> if Mirrormaker replication is set to start at "latest". This bug causes the
> target consumer group's lag for that partition to be negative and breaks
> offset sync for that partition until lag is positive.
> The correct behavior when the source partition is empty would be to set the
> target offset to the translated offset, not literal offset, which in this
> case would always be 0.
> Original email thread on this issue:
> https://lists.apache.org/thread.html/r7c54ee5f57227367b911d4abffa72781772d8dd3b72d75eb65ee19f7%40%3Cusers.kafka.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
