Re: [DISCUSS] Is it a bug that the AdaptiveScheduler does not prioritize releasing TaskManagers during downscaling in Application mode?

Hey All! The main purpose of the adaptive scheduler is to be able to adapt to changing resource availability and requirements. Originally it was designed to work based on resource availability (with reactive scaling) so when we have more resources we scale up, if we have less scale down, at that p

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations! Best, Yuepeng Pan At 2024-11-06 14:08:41, "Xia Sun" wrote: >Congratulations Junrui! > > >Best, >Xia > >Zakelly Lan 于2024年11月6日周三 09:57写道： > >> Congratulations, Junrui! >> >> >> Best, >> Zakelly >> >> On Wed, Nov 6, 2024 at 9:50 AM Feng Jin wrote: >> >> > Congratulations Jun

[DISCUSS] Is it a bug that the AdaptiveScheduler does not prioritize releasing TaskManagers during downscaling in Application mode?

Hi, community. When working on ticket[1] we have received some lively discussions and valuable feedback[2](thanks for Matthias, Rui, Gyula, Maximilian, Tison, etc.), the main issues are that: When the job runs in an application cluster, could the default behavior of AdaptiveScheduler not act

Re: Flink Kubernetes operator

Hey! I think this question should have been sent to the user mailing list instead of dev :) In any case: 1: Yes the autoscaler logic is triggered in the reconcile loop, but that just means that it is triggered every X seconds and only if there is no other thing to do from the operator like perfo

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui! Best, Xia Zakelly Lan 于2024年11月6日周三 09:57写道： > Congratulations, Junrui! > > > Best, > Zakelly > > On Wed, Nov 6, 2024 at 9:50 AM Feng Jin wrote: > > > Congratulations Junrui! > > > > > > Best, > > Feng > > > > > > On Wed, Nov 6, 2024 at 9:36 AM Rui Fan <1996fan...@gmai

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations, Junrui! Best, Zakelly On Wed, Nov 6, 2024 at 9:50 AM Feng Jin wrote: > Congratulations Junrui! > > > Best, > Feng > > > On Wed, Nov 6, 2024 at 9:36 AM Rui Fan <1996fan...@gmail.com> wrote: > > > Congratulations Junrui! > > > > Best, > > Rui > > > > On Wed, Nov 6, 2024 at 9:33 

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui! Best, Feng On Wed, Nov 6, 2024 at 9:36 AM Rui Fan <1996fan...@gmail.com> wrote: > Congratulations Junrui! > > Best, > Rui > > On Wed, Nov 6, 2024 at 9:33 AM Guowei Ma wrote: > > > Congratulations Junrui! > > > > Best, > > Guowei > > > > > > On Wed, Nov 6, 2024 at 9:26 

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui! Best, Rui On Wed, Nov 6, 2024 at 9:33 AM Guowei Ma wrote: > Congratulations Junrui! > > Best, > Guowei > > > On Wed, Nov 6, 2024 at 9:26 AM Shengkai Fang wrote: > > > Congratulations！ > > > > Best, > > Shengkai > > > > Lincoln Lee 于2024年11月6日周三 00:05写道： > > > > > Congr

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui! Best, Guowei On Wed, Nov 6, 2024 at 9:26 AM Shengkai Fang wrote: > Congratulations！ > > Best, > Shengkai > > Lincoln Lee 于2024年11月6日周三 00:05写道： > > > Congratulations Junrui! > > > > > > Best, > > Lincoln Lee > > > > > > Zhanghao Chen 于2024年11月5日周二 23:32写道： > > > > > C

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations！ Best, Shengkai Lincoln Lee 于2024年11月6日周三 00:05写道： > Congratulations Junrui! > > > Best, > Lincoln Lee > > > Zhanghao Chen 于2024年11月5日周二 23:32写道： > > > Congrats, Junrui! > > > > > > Best, > > Zhanghao Chen > > > > From: Zhu Zhu > > Sent: Tuesda

Planned release dates for 1.19.2 and 1.20.1

Hi all , What’s the planned release dates for flink versions 1.19.2 and 1.20.1. Thank you, Prasanna

Flink Kubernetes operator

Hi, I have a few questions about the Flink OSS Kubernetes operator. Apologies if they’re basic. 1. It seems like the auto-scaler logic is triggered as part of the operator reconciliation loop

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #2

Thanks for the question Danny. We are making the major version bump from 4 to 5 because of the following changes: 1/ New connectors (SQS sink, DynamoDB Streams source). We are also adding more functionality to the KDS source based on FLIP-27 interfaces. 2/ Breaking changes to the KDS source via Ta

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #2

Thanks for driving this release Hong. I cannot find the discussion thread, and was wondering what the justification is for a major release this time? What are we breaking? I will take a look at the release artifacts tomorrow. Thanks, Danny On Tue, 5 Nov 2024, 08:06 Yanquan Lv, wrote: > +1 (non

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui! Best, Lincoln Lee Zhanghao Chen 于2024年11月5日周二 23:32写道： > Congrats, Junrui! > > > Best, > Zhanghao Chen > > From: Zhu Zhu > Sent: Tuesday, November 5, 2024 19:59 > To: dev ; Junrui Lee > Subject: [ANNOUNCE] New Apache Flink Committer -

[jira] [Created] (FLINK-36662) Incorrect constant pull up when group keys in aggregate are NULL

lincoln lee created FLINK-36662: --- Summary: Incorrect constant pull up when group keys in aggregate are NULL Key: FLINK-36662 URL: https://issues.apache.org/jira/browse/FLINK-36662 Project: Flink

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congrats, Junrui! Best, Zhanghao Chen From: Zhu Zhu Sent: Tuesday, November 5, 2024 19:59 To: dev ; Junrui Lee Subject: [ANNOUNCE] New Apache Flink Committer - Junrui Li Hi everyone, On behalf of the PMC, I'm happy to announce that Junrui Li has become a new F

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations, Junrui ! Best, Biao Geng Lijie Wang 于2024年11月5日周二 20:20写道： > Congratulations Junrui ! > > Best, > Lijie > > Zhu Zhu 于2024年11月5日周二 20:00写道： > > > Hi everyone, > > > > On behalf of the PMC, I'm happy to announce that Junrui Li has become a > > new Flink Committer! > > > > Junru

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui ! Best, Lijie Zhu Zhu 于2024年11月5日周二 20:00写道： > Hi everyone, > > On behalf of the PMC, I'm happy to announce that Junrui Li has become a > new Flink Committer! > > Junrui has been an active contributor to the Apache Flink project for two > years. He had been the driver and

[ANNOUNCE] New Apache Flink Committer - Junrui Li

Hi everyone, On behalf of the PMC, I'm happy to announce that Junrui Li has become a new Flink Committer! Junrui has been an active contributor to the Apache Flink project for two years. He had been the driver and major developer of 8 FLIPs, contributed 100+ commits with tens of thousands of code

Re: [DISCUSS] Optimize the processing logic of TopNFunction to avoid the expiration of downstream operator states when state TTL is set

Hi, Xuyang, This issue is quite similar to FLINK-33936. Since the inconsistency in state expiration between the rank operator and downstream operators only occurs in specific scenarios, we might need to discuss whether it’s necessary to modify the default behavior. Best, Feng On Mon, Nov 4, 2

Re: [DISCUSS] AdaptiveScheduler slots assignment strategy optimization

Sorry for the inappropriate email. It seems the current discussion has unintentionally misrepresented the issue and its scope. Therefore, the discuss thread is closed now. To keep our focus on the main objective, I’ll start a new discussion thread specifically for this purpose. Thank you for y

Re: [VOTE] Release flink-connector-prometheus v1.0.0, release candidate #1

+1 (non binding) - verified checksums - run an e2e application using the 1.0.0-1.20 artifacts, (including Amazon Managed Prometheus request signer) Best regards Lorenzo On Tue, 5 Nov 2024 at 07:37, Xintong Song wrote: > +1 (binding) > > - verified signatures and checksum > - built from source

Re: [VOTE] Release flink-connector-prometheus v1.0.0, release candidate #1

+1 (non-binding) I've checked: - Review JIRA release notes - Verify hashes and verify signatures - Build success from source with JDK11 & maven3.8.6 - Source code artifacts matching the current release > 2024年11月5日 07:28，Hong Liang 写道： > > Hi everyone, > Please review and vote on release candid

Re: Micro batching with flink

Hey, could you please check if a bucket assigner is already enough? If not, what's missing? FileSink orcSink = FileSink .forBulkFormat(new Path("s3a://mybucket/flink_file_sink_orc_test"), factory) .withBucketAssigner(new DateTimeBucketAssigner<>("'dt='MMdd/'hour='HH",

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #2

+1 (non-binding) Thanks for Hong’s explanation. I'm just making sure we haven't missed any previous discussions. > 2024年11月5日 15:51，Hong Liang 写道： > > Hi Yanquan, > Thanks for checking in - I did not send an email about RC1, because I built > it, then realized we wanted to include an addition