That makes sense. Thanks On Thu, Aug 20, 2020 at 7:45 PM Piotr Nowojski <pnowoj...@apache.org> wrote:
> Hi Ori, > > No. Flink does it differently. Operators that are keeping track of late > events, are remembering the latest watermark. If a new element arrives with > even time lower compared to the latest watermark, it is marked as a late > event [1] > > Piotrek > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.11/concepts/timely-stream-processing.html#lateness > > czw., 20 sie 2020 o 17:13 Ori Popowski <ori....@gmail.com> napisał(a): > >> In the documentation >> <https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#allowed-lateness> >> it states that: >> >> *[…], Flink keeps the state of windows until their allowed lateness >> expires. Once this happens, Flink removes the window and deletes its state, >> as also described in the Window Lifecycle >> <https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#window-lifecycle> >> section.* >> >> However, something doesn't make sense to me. >> >> If Flink deletes the window state, then how can it know that subsequent >> events are late? i.e. if the state is deleted, then Flink has no way of >> knowing than an event is late, because it can think it's just a new event, >> unless it keeps track of which keyed windows are closed forever. >> >> Does Flink remember which keyed windows are closed forever? >> >> Thanks. >> >
