[
https://issues.apache.org/jira/browse/KAFKA-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434075#comment-17434075
]
Sagar Rao commented on KAFKA-12960:
-----------------------------------
That’s something which the pr already addresses for persistent store. The
problem or rather the topic of contention has been that the observed stream
time needs to be visible at the metered store level. For which I needed to
introduce a new interface. if we need to do it at the metered store level and
also want this to be automatically available for custom state stores, then it
will mean we will have to expose the observed stream time for in memory
stores. And also for custom stores, we will need to provide the ability to set
and expose it.
> WindowStore and SessionStore do not enforce strict retention time
> -----------------------------------------------------------------
>
> Key: KAFKA-12960
> URL: https://issues.apache.org/jira/browse/KAFKA-12960
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Sagar Rao
> Priority: Major
>
> WindowedStore and SessionStore do not implement a strict retention time in
> general. We should consider to make retention time strict: even if we still
> have some record in the store (due to the segmented implementation), we might
> want to filter expired records on-read. This might benefit PAPI users.
> Atm, InMemoryWindow store does already enforce a strict retention time.
> As an alternative, we could also inject such a filter in the wrapping
> `MeteredStore` – this might lift the burden from users who implement a custom
> state store.
> As an alternative, we could change all DSL operators to verify if data from a
> state store is already expired or not. It might be better to push this
> responsibility into the stores though.
> It's especially an issue for stream-stream joins, because the operator relies
> on the retention time to implement it's grace period.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
