Lucas Bradstreet created KAFKA-12177:
----------------------------------------
Summary: Retention is not idempotent
Key: KAFKA-12177
URL: https://issues.apache.org/jira/browse/KAFKA-12177
Project: Kafka
Issue Type: Improvement
Reporter: Lucas Bradstreet
Kafka today applies retention in the following order:
# Time
# Size
# Log start offset
Today it is possible for a segment with offsets less than the log start offset
to contain data that is not deletable due to time retention. This means that
it's possible for log start offset retention to unblock further deletions as a
result of time based retention. Note that this does require a case where the
max timestamp for each segment increases, decreases and then increases again.
Even so it would be nice to make retention idempotent by applying log start
offset retention first, followed by size and time. This would also be
potentially cheaper to perform as neither log start offset and size retention
require the maxTimestamp for a segment to be loaded from disk after a broker
restart.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
