Re: [DISCUSS] FLIP-542: Support DISTRIBUTED BY/INTO for Materialized tables

2025-08-27 Thread Sergey Nuyanzin
Hi Ron thank you, updated. Thanks everyone if there is no objections I'm going to start voting next week On Wed, Aug 20, 2025 at 3:43 AM Ron Liu wrote: > > Hi, Sergey. > > Thanks for the FLIP to improve the materialized table ddl. The proposal > looks good to me overall. > > Just one minor comm

Re: [DISCUSS] FLIP-543: Support Customized Autoscale Algorithm

2025-08-27 Thread Pradeepta Choudhury
Hello Peter, To start with, great initiative! But I echo the same concern raised about creating too many extension points can compromise the autoscaler functionality. When we proposed FLIP-514 [1] and a custom evaluator, the aim was twofold: provide the required extension point and ship practi

[DISCUSS] FLIP-430: Remote Compaction For Disaggregated State

2025-08-27 Thread Han Yin
Hi everyone, I would like to open a discussion on introducing remote compaction for disaggregated state[1]. Flink state backends rely on LSM-Trees for large-scale storage, with file compaction executed locally in TaskManager background threads. This co-location creates local resource contentio

[DISCUSS] FLIP - 544 SinkUpsertMaterializer V2

2025-08-27 Thread Roman Khachatryan
Hi everyone, I would like to start a discussion about FLIP-544 SinkUpsertMaterializer V2 [1]. SinkUpsertMaterializer is an operator in Flink that reconciles out of order changelog events before sending them to an upsert sink. In some cases (that we see in our production), performance of this oper

[jira] [Created] (FLINK-38290) Application cluster: FINISHED FlinkDeployment falls back to RECONCILING if JM restarts

2025-08-27 Thread Urs Schoenenberger (Jira)
Urs Schoenenberger created FLINK-38290: -- Summary: Application cluster: FINISHED FlinkDeployment falls back to RECONCILING if JM restarts Key: FLINK-38290 URL: https://issues.apache.org/jira/browse/FLINK-38290

Re: [DISCUSS] FLIP-543: Support Customized Autoscale Algorithm

2025-08-27 Thread ctrlaltdilj
A pre/post scaling hook maybe helpful here, I think if done right it could cover a couple scenarios like having a plugin influence the amount of scaling that happens

[jira] [Created] (FLINK-38289) Update Flink Connector Kafka to use Flink 2.1

2025-08-27 Thread Thomas Cooper (Jira)
Thomas Cooper created FLINK-38289: - Summary: Update Flink Connector Kafka to use Flink 2.1 Key: FLINK-38289 URL: https://issues.apache.org/jira/browse/FLINK-38289 Project: Flink Issue Type: I

Re: [DISCUSS] FLIP-490: Enhanced Job History Retention Policies for HistoryServer

2025-08-27 Thread Yuepeng Pan
Hi, community. This discussion has been ongoing for two weeks, and I really appreciate the attention and support from the developers! If there is no further feedback this week, I will initiate a vote next week. Best regards, Yuepeng Pan Replied Message | From | Yuepeng Pan | | Date |