suzhiking opened a new pull request, #23183: URL: https://github.com/apache/kafka/pull/23183
Offsets committed without a leader epoch (e.g. via `new OffsetAndMetadata(offset)`) cannot be validated with the OffsetsForLeaderEpoch API when a consumer resumes from them. If such an offset no longer exists in the partition — for example because the log was truncated after a partition leader change — the consumer gets OFFSET_OUT_OF_RANGE on fetch and falls back to the `auto.offset.reset` policy, which can cause large-scale duplicate processing (`earliest`) or data loss (`latest`). This surfaced in [KAFKA-19902](https://issues.apache.org/jira/browse/KAFKA-19902), where an application committed epoch-less offsets from a rebalance listener and a leader change plus log truncation later forced a full reset to earliest. The existing Javadoc recommended including the leader epoch but never explained the consequences of omitting it, and the `OffsetAndMetadata` constructors did not mention the field at all. It also described the epoch as "commit metadata", which invites confusion with the metadata string. Javadoc-only change, no behavior change: - `OffsetAndMetadata`: document the `leaderEpoch` field's role in position validation at the class level, warn on the two constructors that leave it empty, and recommend the three-arg constructor with `record.leaderEpoch()`. - `KafkaConsumer` "Manual Offset Control" section: replace the bare recommendation with an explanation of what the committed epoch enables and what happens without it, and note that `ConsumerRecords.nextOffsets()` already includes it. - `commitSync(Map)` / `commitSync(Map, Duration)` / `commitAsync(Map, callback)`: state the consequence and link to the class-level section. - `seek(TopicPartition, long)`: note that it clears the position's leader epoch, so offsets committed before any further records are consumed lack an epoch; point to the `seek(TopicPartition, OffsetAndMetadata)` overload. Verified with `./gradlew :clients:javadoc`; no tests needed for a documentation-only change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
