shangxinli commented on PR #18125: URL: https://github.com/apache/hudi/pull/18125#issuecomment-3993886401
**Re: partition disappearing between commits in `calculateKafkaOffsetDifference`** Good question — yes, it's possible a partition exists in the previous commit but not in the current one (e.g., topic reconfiguration, consumer rebalance). The offset diff contribution for such a partition is correctly 0 — no records were consumed from it in this batch, so there's nothing to count. However, a disappearing partition may indicate an issue worth investigating, so I added a `LOG.warn` that flags each such partition along with its previous offset. No functional change to the diff calculation itself — we still only iterate over `currentOffsets` since those are the partitions that actually produced records in this batch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
