RE: Can flink dynamically detect high load and increase the job parallelism automatically?

2022-09-14 Thread Erez Yaakov
clarifications. It's really helpful! From: zhanghao.c...@outlook.com Sent: Tuesday, September 13, 2022 4:30 AM To: Erez Yaakov ; user@flink.apache.org Subject: Re: Can flink dynamically detect high load and increase the job parallelism automatically? EXTERNAL EMAIL Hi Erez, Unfortunately

RE: Can flink dynamically detect high load and increase the job parallelism automatically?

2022-09-13 Thread Erez Yaakov
Thanks Zhanghao for your reply. Do you know approximately what are the future plans regarding maturing this feature and having it available in application mode on native k8s? Thanks, Erez From: zhanghao.c...@outlook.com Sent: Tuesday, September 13, 2022 4:30 AM To: Erez Yaakov ; user

Can flink dynamically detect high load and increase the job parallelism automatically?

2022-09-12 Thread Erez Yaakov
Hi, When running a streaming job that uses a kafka source, is it possible (without reactive mode) for flink to dynamically detect high load (high consumers lag, high cpu usage...) and increase the job parallelism automatically? I am running flink streaming job on an application mode cluster