Re: How windowing is implemented on Flink runner
Hi, Wanted to follow up as I did have similar case. So this means it is ok for Beam to use Sliding window of 1 day with 1 sec period (with using different trigger than after watermark to avoid outputting data from every window) and there is no additional performance penalty (duplicating input messages for storage or cpu for resolving windows)? Interesting from both Flink and Dataflow perspective (both Python and Java). I ended up implementing the logic with Beam state and timers (which is quite performant and readable) but also interested in other possibilities. Best Wiśniowski Piort On 12.06.2024 21:50, Ruben Vargas wrote: I imagined it but wasn't sure! Thanks for the clarification! On Wed, Jun 12, 2024 at 1:42 PM Robert Bradshaw via user wrote: Beam implements Windowing itself (via state and timers) rather than deferring to Flink's implementation. On Wed, Jun 12, 2024 at 11:55 AM Ruben Vargas wrote: Hello guys May be a silly question, But in the Flink runner, the window implementation uses the Flink windowing? Does that mean the runner will have performance issues like Flink itself? see this: https://issues.apache.org/jira/browse/FLINK-7001 I'm asking because I see the issue, it mentions different concepts that Beam already handles at the API level. So my suspicion is that the Beam model handles windowing a little differently from the pure Flink app. But I'm not sure.. Regards.
Re: How windowing is implemented on Flink runner
I imagined it but wasn't sure! Thanks for the clarification! On Wed, Jun 12, 2024 at 1:42 PM Robert Bradshaw via user wrote: > > Beam implements Windowing itself (via state and timers) rather than > deferring to Flink's implementation. > > On Wed, Jun 12, 2024 at 11:55 AM Ruben Vargas wrote: > > > > Hello guys > > > > May be a silly question, > > > > But in the Flink runner, the window implementation uses the Flink > > windowing? Does that mean the runner will have performance issues like > > Flink itself? see this: > > https://issues.apache.org/jira/browse/FLINK-7001 > > > > I'm asking because I see the issue, it mentions different concepts > > that Beam already handles at the API level. So my suspicion is that > > the Beam model handles windowing a little differently from the pure > > Flink app. But I'm not sure.. > > > > > > Regards.
Re: How windowing is implemented on Flink runner
Beam implements Windowing itself (via state and timers) rather than deferring to Flink's implementation. On Wed, Jun 12, 2024 at 11:55 AM Ruben Vargas wrote: > > Hello guys > > May be a silly question, > > But in the Flink runner, the window implementation uses the Flink > windowing? Does that mean the runner will have performance issues like > Flink itself? see this: > https://issues.apache.org/jira/browse/FLINK-7001 > > I'm asking because I see the issue, it mentions different concepts > that Beam already handles at the API level. So my suspicion is that > the Beam model handles windowing a little differently from the pure > Flink app. But I'm not sure.. > > > Regards.
How windowing is implemented on Flink runner
Hello guys May be a silly question, But in the Flink runner, the window implementation uses the Flink windowing? Does that mean the runner will have performance issues like Flink itself? see this: https://issues.apache.org/jira/browse/FLINK-7001 I'm asking because I see the issue, it mentions different concepts that Beam already handles at the API level. So my suspicion is that the Beam model handles windowing a little differently from the pure Flink app. But I'm not sure.. Regards.
