Doyeon Kim created KAFKA-21272:
----------------------------------
Summary: localLogStartOffset is not advanced when remote storage
is enabled on an existing topic
Key: KAFKA-21272
URL: https://issues.apache.org/jira/browse/KAFKA-21272
Project: Kafka
Issue Type: Bug
Reporter: Doyeon Kim
Assignee: Doyeon Kim
While remote storage is disabled for a topic, maybeIncrementLogStartOffset
advances only logStartOffset, so retention and DeleteRecords leave
localLogStartOffset behind. When remote.storage.enable is then set to true,
nothing advances it. localLogStartOffset stays below logStartOffset until the
broker restarts or an uploaded segment is deleted locally.
In the meantime, ListOffsets returns wrong values:
* EARLIEST_LOCAL_TIMESTAMP returns the stale localLogStartOffset, which is
below logStartOffset.
* EARLIEST_PENDING_UPLOAD_TIMESTAMP returns -1 although nothing has been
uploaded yet.
Repro (unit test against UnifiedLog on trunk): segments [0,5) [5,10) [10,15)
with remote storage disabled, DeleteRecords to 7, then enable remote storage.
Result: logStartOffset=7, localLogStartOffset=0, EARLIEST_LOCAL=0,
EARLIEST_PENDING_UPLOAD=-1. After a restart, both return 7.
Disabling remote storage already handles the opposite direction: ConfigHandler
advances logStartOffset to localLogStartOffset. Enabling it could do the same
for localLogStartOffset.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
