Shantanu Kawale created KAFKA-21317:
---------------------------------------
Summary: Timestamp lookup can miss valid offsets after the log
start offset advances
Key: KAFKA-21317
URL: https://issues.apache.org/jira/browse/KAFKA-21317
Project: Kafka
Issue Type: Bug
Components: core
Reporter: Shantanu Kawale
UnifiedLog.searchOffsetInLocalLog returns immediately from the first segment
whose largestTimestamp meets the requested timestamp, even when that segment's
findOffsetByTimestamp returns empty.
This can incorrectly return no result for a timestamp lookup when the log start
offset has advanced into a segment and a later segment contains a valid match.
*Reproduction*
Confirmed on Apache Kafka trunk commit:
8e46092edac3e837972ee13b40dcc149e7113ae7
(4.5.0-SNAPSHOT, tested October 20, 2026)
1. Append offset 0 at timestamp T and offset 1 at timestamp T-1 into the first
segment.
2. Roll the log and append offset 2 at timestamp T into the second segment.
3. Advance the high watermark to the log end.
4. Advance the log start offset to 1, as with DeleteRecords, leaving the first
segment physically present.
5. Query timestamp T.
Expected: timestamp T at offset 2.
Actual: an empty result.
*Root cause*
The first segment remains eligible because its cached largest timestamp comes
from offset 0. LogSegment correctly excludes that record because it is below
the log start offset. UnifiedLog then returns the empty result without
searching the second segment.
*Proposed fix*
Continue searching eligible segments until a lookup returns a result. Use
in-memory segment base offsets to skip obsolete segments without reading
segment tails.
*Validation*
- The new regression fails with trunk production code unchanged.
- The same regression passes with the fix applied.
- All 243 tests in UnifiedLogTest and LogSegmentTest pass.
- Storage Spotless, Checkstyle, and SpotBugs checks pass.
*Proposed patch:*
https://github.com/shantanukawale-stripe/kafka/tree/fix-unified-log-timestamp-search
Related issues
KAFKA-10875 reports similar symptoms, but equivalence has not been established.
KAFKA-21204 concerns the upper fetchable offset bound, not this cross-segment
search failure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
