Stanislav Kozlovski created KAFKA-7910:
------------------------------------------
Summary: Document retention.ms behavior with record timestamp
Key: KAFKA-7910
URL: https://issues.apache.org/jira/browse/KAFKA-7910
Project: Kafka
Issue Type: Improvement
Reporter: Stanislav Kozlovski
It is intuitive to believe that `log.retention.ms` starts applying once a log
file is closed.
The documentation says:
> This configuration controls the maximum time we will retain a log before we
>will discard old log segments to free up space if we are using the "delete"
>retention policy.
Yet, the actual behavior is that we take into account the largest timestamp of
that segment file
([https://github.com/apache/kafka/blob/4cdbb3e5c19142d118f0f3999dd3e21deccb3643/core/src/main/scala/kafka/log/Log.scala#L1246)|https://github.com/apache/kafka/blob/4cdbb3e5c19142d118f0f3999dd3e21deccb3643/core/src/main/scala/kafka/log/Log.scala#L1246).]
and then consider `retention.ms` on top of that.
This means that if Kafka is configured with
`log.message.timestamp.type=CreateTime` (as it is by default), any records that
have a future timestamp set by the producer will not get deleted as expected by
the initial intuition (and documentation) of the `log.retention.ms`.
We should document the behavior of `retention.ms` with the record timestamp.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
