Beam High Priority Issue Report (54)

2024-05-07 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/31122 The PostCommit Go V

Job execution starts very slowly on Flink server.

2024-05-07 Thread Balogh , György
Hi, We are switching from embedded flink runner to filink server. The embedded flink runner works fine, jobs start instantly. However when we use the flink server we are experiencing about 30 seconds of idle time (no significant cpu, io load) before we see the job in the flink. We are targeting at

Re: Job execution starts very slowly on Flink server.

2024-05-07 Thread Steve Niemitz via dev
This sounds about right to me. When I was working on getting flink running this was one of the biggest pain points, the upload/download process is painfully slow. We had implemented a jar cache in flink to improve this (and stopped using fat jars) https://github.com/twitter-forks/flink/commit/01f

Chronon + Beam

2024-05-07 Thread Varant Zanoyan
Hi Beam community, Nikhil and I are two of the authors of the Chronon project, maintained by Airbnb and Stripe. We're planning on adding an Enrichment Handler for Beam to Chronon soon, so we wanted to reach out and see if anyone in this community would be interested in us