I haven't thought through this, but I'd be interested in seeing how this would affect windows that use Beam's KafkaIO withLogAppendTime (which assume LogAppendTime is monotonic afaik). It makes me wonder if that works correctly today.
-Jon > On Sep 6, 2026, at 11:17 AM, Chia-Ping Tsai <[email protected]> wrote: > > Hi Alexander > > It's interesting and worth mulling it over. Would you mind opening a jira and > KIP to propose this change? > > Best, > Chia-Ping > > On 2026/08/18 08:34:09 Alexander Neubeck wrote: >> Hi Kafka developers, >> >> KIP-32 describes LogAppendTime as monotonically increasing. However, the >> current implementation stamps records using the active leader’s local clock. >> >> After a leader election, the new leader’s clock may be behind the previous >> leader’s clock. In that case, newly appended records can receive a lower >> LogAppendTime than preceding records in the partition. >> >> A possible fix is for each partition leader to select: >> >> Plain text >> >> max(current broker time, previous LogAppendTime) >> >> >> >> The previous value can be recovered from the replicated log tail and >> maintained across follower appends, restart, truncation, and leader >> promotion. This guarantees nondecreasing timestamps without forcing a >> one-millisecond increase for every append. >> >> Would this be considered a correctness fix completing KIP-32’s intended >> monotonicity guarantee, or would the observable timestamp behavior require a >> follow-up KIP? >> >> KIP-32: >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message >> >> Current assignment logic: >> >> https://github.com/apache/kafka/blob/trunk/storage/src/main/java/org/apache/kafka/storage/internals/log/LogValidator.java >> >> Thanks, >> >> Alexander >>
