Re: GroupIntoBatches not working on Flink?

2022-07-26 Thread Reuven Lax via user
This might be a bug in the Flink runner, because it is implemented here . On Tue, Jul 26, 2022 at 9:14 AM Cristian

GroupIntoBatches not working on Flink?

2022-07-26 Thread Cristian Constantinescu
Hi everyone, Quick question about GroupIntoBatches. When running on Flink, eventually it hits an unsupported exception "Canceling a timer by ID is not yet supported." on this line [1]. The source inputs are AVRO files for testing (batch) but will use kafka topics (streaming) when deployed. This

[Help Needed] Flink JVM Metaspace OoM running Beam Python pipelines

2022-07-26 Thread Nianlong "Lennon" Yu
Hi folks, I'm testing running Beam Python pipelines on Flink but I'm running into consistent Flink JVM metaspace OoM issues. The symptom is that Flink's task manager's JVM metaspace usage will monotonically increase with each run of the Beam Python pipeline, and never goes down, eventually