Hi, We are switching from embedded flink runner to filink server. The embedded flink runner works fine, jobs start instantly. However when we use the flink server we are experiencing about 30 seconds of idle time (no significant cpu, io load) before we see the job in the flink. We are targeting at most 2 seconds overhead to run ad-hoc query pipelines. We did some investigation and saw that the fat jar is about 150MB that was sent from the beam to the flink cluster. Still this does not justify the 30 sec. We are trying to deploy dependencies to the flink workers and decrease the jar size. It's not clear at the moment what causes this delay.
Flink version: 1.16 Beam version: 2.55.1 Thank you, Gyorgy -- György Balogh CTO E gyorgy.bal...@ultinous.com <zsolt.sala...@ultinous.com> M +36 30 270 8342 <+36%2030%20270%208342> A HU, 1117 Budapest, Budafoki út 209. W www.ultinous.com
