gy setup?
>
>
>
> Just as said before, ignoring-late-events is a default strategy, that can
> be adjusted by means of a custom window trigger which trades off between
> latency, state size, correctness of the final results.
>
>
>
> Thias
>
>
>
> *From
.
Thias
From: John Smith
Sent: Freitag, 26. November 2021 17:17
To: Schwalbe Matthias
Cc: Caizhi Weng ; user
Subject: Re: Windows and data loss.
Or as an example we have a 5 minutes window and lateness of 5 minutes.
We have the following events in the logs
10:00:01 PM > Already pus
og: [2] [3]
>>
>> I believe his code uses Beam, but the same ideas can be implemented
>> directly in Flink API
>>
>>
>>
>> [1] https://www.oreilly.com/library/view/streaming-systems/9781491983867/
>>
>> [2] https://www.oreilly.com/radar/the-wo
> (network interrupt, datacenter flooded etc. ). With processing time
> events count into the time window when processed, with event time they
> count into the time window when originally created a the source … even if
> processed much later …
>
>
>
> Thias
>
>
&
] https://www.oreilly.com/radar/the-world-beyond-batch-streaming-102/
… happy to discuss further
Thias
From: John Smith
Sent: Freitag, 26. November 2021 14:09
To: Schwalbe Matthias
Cc: Caizhi Weng ; user
Subject: Re: Windows and data loss.
But if we use event time, if a failure happens
gt;
>
>
> *From:* John Smith
> *Sent:* Freitag, 26. November 2021 02:55
> *To:* Schwalbe Matthias
> *Cc:* Caizhi Weng ; user
> *Subject:* Re: Windows and data loss.
>
>
>
> Well what I'm thinking for 100% accuracy no data loss just to base the
> count on process
To: Schwalbe Matthias
Cc: Caizhi Weng ; user
Subject: Re: Windows and data loss.
Well what I'm thinking for 100% accuracy no data loss just to base the count on
processing time. So whatever arrives in that window is counted. If I get some
events of the "current" window late a
l extend the window processing (or is that done in the trigger … I
>don’t remember right know)
>- overall window state grows, if you extend window processing to after
>it is finished …
>
>
>
> Hope this helps
>
>
>
> Thias
>
>
>
> *From:* Caiz
To: John Smith
Cc: user
Subject: Re: Windows and data loss.
Hi!
Are you using the datastream API or the table / SQL API? I don't know if
datastream API has this functionality, but in table / SQL API we have the
following configurations [1].
* table.exec.emit.late-fire.enabled: Emit window
Thanks. Using, data streaming.
On Wed, 24 Nov 2021 at 20:56, Caizhi Weng wrote:
> Hi!
>
> Are you using the datastream API or the table / SQL API? I don't know if
> datastream API has this functionality, but in table / SQL API we have the
> following configurations [1].
>
>-
Hi!
Are you using the datastream API or the table / SQL API? I don't know if
datastream API has this functionality, but in table / SQL API we have the
following configurations [1].
- table.exec.emit.late-fire.enabled: Emit window results for late
records;
-
11 matches
Mail list logo
