Jason Gustafson created KAFKA-9196:
--------------------------------------
Summary: Records exposed before advancement of high watermark
after segment roll
Key: KAFKA-9196
URL: https://issues.apache.org/jira/browse/KAFKA-9196
Project: Kafka
Issue Type: Bug
Reporter: Jason Gustafson
Assignee: Jason Gustafson
Fix For: 2.4.0
We cache the segment position of the high watermark and last stable offset
inside `Log`. There is no logic currently to update the cached position when
the segment rolls.
Suppose we have a log with one segment (0.log). We write 5 records and update
the high watermark to match the log end offset. The cached segment position
will be something like LogOffsetMetadata(offset=5, segment=0, position=100).
Now suppose we roll to segment 5.log and write some new data. If a consumer
fetches from offset 5, then the current fetch logic will find segment 5.log and
incorrectly use position 100 in this segment. The result is that data from the
new segment gets exposed prematurely.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
