Guozhang Wang created KAFKA-6347:
------------------------------------
Summary: Starting offset breach based log segment deletion never
considers active segment
Key: KAFKA-6347
URL: https://issues.apache.org/jira/browse/KAFKA-6347
Project: Kafka
Issue Type: Improvement
Components: core
Reporter: Guozhang Wang
This observation is related to KIP-107: in
{{Log#deleteLogStartOffsetBreachedSegments}}, we designed the predicate to
enforce that the current segment can only be considered for deletion, when it
1) has the next segment, 2) the next segment's starting offset is no larger
than the log start offset.
{code}
def shouldDelete(segment: LogSegment, nextSegmentOpt: Option[LogSegment]) =
nextSegmentOpt.exists(_.baseOffset <= logStartOffset)
{code}
This means that, the current active segment would never be considered for
purging. Normally this is OK given that we will eventually roll out a new
segment. However, with the default size of segment of 1GB it means that with
the purge data API we are in the worst case not being able to purge the data
for up to 1GB per topic partition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
