Thanks for the KIP Sophie. It make total sense to get rid of default grace period of 24h.
Some questions/comments: (1) Is there any particular reason why we want to remove `grace(Duration)` method? (2) About `SlidingWindows#withTimeDifferenceAndGrace` -- personally I think it's worth to clean it up right now -- given that sliding windows are rather new the "splash radius" should be small. (3) Some nits on wording: > This config determines how long after a window closes any new data will still > be processed Should be "after a window ends" -- a window is closed after grace period passed. > one which indicates to use no grace period and not handle out-of-order data Seems strictly not correct -- if there is a window from 0 to 100 and you get record with ts 99,98,97,...,0 all but the first of those records are out-of-order but they are still processed even with a grace period of zero. Maybe better: "one which indicate to use no grace period and close the window immediately when the window ends." > and make a conscious decision to skip the grace period and drop out-of-order > records, Maybe better: "and make a conscious decision to skip the grace period and close a window immediately" -Matthias On 3/31/21 5:02 PM, Guozhang Wang wrote: > Hello Sophie, > > I agree that the old 24-hour grace period should be updated, and I also > think now it is a better idea to make the grace period "mandatory" from the > API names since it is a very important concept and hence worth emphasizing > to users up front. > > Guozhang > > On Wed, Mar 31, 2021 at 1:58 PM John Roesler <vvcep...@apache.org> wrote: > >> Thanks for bringing this up, Sophie! >> >> This has indeed been a pain point for a lot of people. >> >> It's a really thorny issue with no obvious "right" solution. >> I think your proposal is a good one. >> >> Thanks, >> -John >> >> On Wed, 2021-03-31 at 13:28 -0700, Sophie Blee-Goldman >> wrote: >>> Hey all, >>> >>> It's finally time to reconsider the default grace period in Kafka >> Streams, >>> and hopefully save a lot of suppression users from the pain of figuring >> out >>> why their results don't show up until 24 hours later. Please check out >> the >>> proposal and let me know what you think. >>> >>> KIP: >>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-633%3A+Drop+24+hour+default+of+grace+period+in+Streams >>> < >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-633%3A+Drop+24hr+default+grace+period >>> >>> >>> JIRA: https://issues.apache.org/jira/browse/KAFKA-8613 >>> >>> Cheers, >>> Sophie >> >> >> >
