[
https://issues.apache.org/jira/browse/KAFKA-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511762#comment-16511762
]
ASF GitHub Bot commented on KAFKA-6978:
---------------------------------------
vvcephei opened a new pull request #5218: KAFKA-6978: make window retention
time strict
URL: https://github.com/apache/kafka/pull/5218
Enforce window retention times strictly:
* records for windows that are expired get dropped
* queries for timestamps old enough to be expired immediately answered with
`null`
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Make Streams Window retention time strict
> -----------------------------------------
>
> Key: KAFKA-6978
> URL: https://issues.apache.org/jira/browse/KAFKA-6978
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: John Roesler
> Assignee: John Roesler
> Priority: Major
>
> Currently, the configured retention time for windows is a lower bound. We
> actually keep the window around until it's time to roll a new segment. At
> that time, we drop all windows in the oldest segment.
> As long as a window is still in a segment, we will continue to add
> late-arriving records to it and also serve IQ queries from it. This is sort
> of nice, because it makes optimistic use of the fact that the windows live
> for some time after their retention expires. However, it is also a source of
> (apparent) non-determinism, and it's arguably better for programability if we
> adhere strictly to the configured constraints.
> Therefore, the new behavior will be:
> * once the retention time for a window passes, Streams will drop any
> later-arriving records (with a warning log and a metric)
> * likewise, IQ will first check whether the window is younger than its
> retention time before answering queries.
> No changes need to be made to the underlying segment management, this is
> purely to make the behavior more strict wrt the configuration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
