[
https://issues.apache.org/jira/browse/KAFKA-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16933864#comment-16933864
]
Matthias J. Sax commented on KAFKA-8924:
----------------------------------------
I have to be more precise. If `-1` is set, grace period default to the
configured retention period that is 24h by default... \cc [~vvcephei] (can you
confirm?)
{quote}To be honest my first thought was that if I do not specify the grace
period it is 0. That makes the most sense to me and that is why I put it as a
proposal.
{quote}
We all agreed on that during the KIP discussion, however, setting grace period
to zero by default would have been a backward incompatible change (and it still
would be – maybe we could do it in a major release). Before we introduced grace
period, there was only retention time with a default of 24h. Retention time was
use to define two things: how long data is stored and how long out-of-order
data is accepted. When we introduce grace period, to be backward compatibly
default grace period could not be set to zero. Note that if you don't use
suppress(), it just works fine... If we set grace period to zero by default, we
would have altered the semantics of existing applications that don't use
suppress() and hence, people would not have been able to upgrade without code
change.
{quote}Why would someone like the window to never close in the first place?
{quote}
If you don't use suppress(), there is not really a notion of closing a window.
Thus, it depends on the use case and desired application semantics.
The tricky part here is, that one configures grace period on the window, and
this configuration affects the downstream suppress() operator. The dependency
between both is the root cause of the issue. When we introduces suppress() and
grace period we knew about this issue and we still want to fix it if possible –
it's just not straight forward to fix. In an ideal case, grace period would
default to the retention time if no downstream suppress() operator is present,
and would default to zero if and only if `suppress()` is present *and* is
configured to only emit when the window closes. However, such a context
dependent default behavior might be even more confusing to users...
> Default grace period (-1) of TimeWindows causes suppress to never emit events
> ------------------------------------------------------------------------------
>
> Key: KAFKA-8924
> URL: https://issues.apache.org/jira/browse/KAFKA-8924
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.3.0
> Reporter: Michał
> Priority: Major
>
> h2. Problem
> The default creation of TimeWindows, like
> {code}
> TimeWindows.of(ofMillis(xxx))
> {code}
> calls an internal constructor
> {code}
> return new TimeWindows(sizeMs, sizeMs, -1, DEFAULT_RETENTION_MS);
> {code}
> And the *-1* parameter is the default grace period which I think is here for
> backward compatibility
> {code}
> @SuppressWarnings("deprecation") // continuing to support
> Windows#maintainMs/segmentInterval in fallback mode
> @Override
> public long gracePeriodMs() {
> // NOTE: in the future, when we remove maintainMs,
> // we should default the grace period to 24h to maintain the default
> behavior,
> // or we can default to (24h - size) if you want to be super accurate.
> return graceMs != -1 ? graceMs : maintainMs() - size();
> }
> {code}
> The problem is that if you use a TimeWindows with gracePeriod of *-1*
> together with suppress *untilWindowCloses*, it never emits an event.
> You can check the Suppress tests
> (SuppressScenarioTest.shouldSupportFinalResultsForTimeWindows), where
> [~vvcephei] was (maybe) aware of that and all the scenarios specify the
> gracePeriod.
> I will add a test without it on my branch and it will fail.
> The test:
> https://github.com/atais/kafka/commit/221a04dc40d636ffe93a0ad95dfc6bcad653f4db
>
> h2. Now what can be done
> One easy fix would be to change the default value to 0, which works fine for
> me in my project, however, I am not aware of the impact it would have done
> due to the changes in the *gracePeriodMs* method mentioned before.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
