Stanislav Kozlovski created KAFKA-7910: ------------------------------------------
Summary: Document retention.ms behavior with record timestamp Key: KAFKA-7910 URL: https://issues.apache.org/jira/browse/KAFKA-7910 Project: Kafka Issue Type: Improvement Reporter: Stanislav Kozlovski It is intuitive to believe that `log.retention.ms` starts applying once a log file is closed. The documentation says: > This configuration controls the maximum time we will retain a log before we >will discard old log segments to free up space if we are using the "delete" >retention policy. Yet, the actual behavior is that we take into account the largest timestamp of that segment file ([https://github.com/apache/kafka/blob/4cdbb3e5c19142d118f0f3999dd3e21deccb3643/core/src/main/scala/kafka/log/Log.scala#L1246)|https://github.com/apache/kafka/blob/4cdbb3e5c19142d118f0f3999dd3e21deccb3643/core/src/main/scala/kafka/log/Log.scala#L1246).] and then consider `retention.ms` on top of that. This means that if Kafka is configured with `log.message.timestamp.type=CreateTime` (as it is by default), any records that have a future timestamp set by the producer will not get deleted as expected by the initial intuition (and documentation) of the `log.retention.ms`. We should document the behavior of `retention.ms` with the record timestamp. -- This message was sent by Atlassian JIRA (v7.6.3#76005)